Prescription opioid laws and opioid dispensing in U.S. counties: Identifying salient law provisions with machine learning

Abstract

Background.

Hundreds of laws aimed at reducing inappropriate prescription opioid dispensing have been implemented in the United States, yet heterogeneity in provisions and their simultaneous implementation have complicated evaluation of impacts. We apply a hypothesis-generating, multi-stage, machine learning approach to identify salient law provisions and combinations associated with dispensing rates to test in future research.

Methods.

Using 162 prescription opioid law provisions capturing prescription drug monitoring program (PDMP) access, reporting and administration features, pain management clinic provisions, and prescription opioid limits, we used regularization approaches and random forest models to identify laws most predictive of county-level and high-dose dispensing. We stratified analyses by overdose epidemic phases—the prescription opioid phase (2006–2009), heroin phase (2010–2012), and fentanyl phase (2013–2016)—to further explore pattern shifts over time.

Results.

PDMP patient data access provisions most consistently predicted high dispensing and high-dose dispensing counties. Pain management clinic-related provisions did not generally predict dispensing measures in the prescription opioid phase but became more discriminant of high dispensing and high-dose dispensing counties over time, especially in the fentanyl period. Predictive performance across models was poor, suggesting prescription opioid laws alone do not strongly predict dispensing.

Conclusions.

Our systematic analysis of 162 law provisions identified patient data access and several pain management clinic provisions as predictive of county prescription opioid dispensing patterns. Future research employing other types of study designs is needed to test these provisions’ causal relationships with inappropriate dispensing, and to examine potential interactions between PDMP access and pain management clinic provisions.

INTRODUCTION

In the past 15 years, in response to the United States (U.S.) opioid crisis, states and the federal government have implemented numerous laws regulating opioid prescribing and dispensing.^1,2 These include prescription drug monitoring programs (PDMPs), pain management clinic laws, and limits on initial opioid prescriptions, among others.^2–7 While prior studies have focused on presence or absence of laws, all laws in each category are not necessarily created equal, and specific provision components may influence effectiveness. There is a critical need for better evidence on how law variations might affect opioid-related outcomes.

Most current literature has focused on PDMPs and adverse opioid-related outcomes.^4,8–12 More comprehensive PDMPs (e.g., must query requirements, etc.) are associated with reduced prescribing and dispensing, increased treatment admissions for opioid use disorder, and some reduction in prescription opioid overdoses; however, some types of PDMPs have been associated with heroin overdose increases.^{5,10,11,13–18} Several single-state analyses show that opioid prescriptions and overdose deaths declined after the enactment of pain management clinic or prescription limits,^19–25 and that joint implementation of comprehensive PDMPs and pain management clinics may reduce opioid prescriptions and overdose deaths.^5,14,26 Other studies have reported no meaningful effects of these laws.^{24,25,27–32}

There are at least two major challenges to evaluating the impacts of prescription opioid laws on opioid dispensing. First, states often adopt widely different versions of the same general type of law, making it particularly important to classify laws beyond binary presence or absence, and to examine the specific provisions that make these laws more or less effective. As noted, provisions requiring prescribers to check the PDMP before writing prescriptions or mandating provider registration are associated with increases in provider use and with some decreases in opioid-related harms.^33–37 However, the role of other PDMP characteristics, including data sharing, funding, administrative structure, and reporting requirements are less well understood, as are the specific characteristics of effective pain management clinic laws.³⁸

Second, states tend to enact multiple law types simultaneously, making it difficult to isolate the effect of any one law, or to examine laws acting synergistically.³⁹ It is possible that some prescription opioid laws or law provisions jointly affect opioid-related outcomes; however, few existing studies consider the potential provision combinations, especially across law domains. Recent reviews of the opioid policy literature highlight the complexity of this shifting state-level policy landscape.^12,40 Their findings underscore the need to develop better approaches for systematically classifying, operationalizing, and exploring heterogenous and concurrent policy provisions in the existing literature.^12,41

Machine learning methods are increasingly being applied to similar high-dimensional data problems, and may offer a complementary approach to other forms of policy analysis. Machine learning methods can be used to as a screening tool to identify particularly high-relevance provisions or provision interactions that require further attention, from within a large set of candidate law exposures. Analogous machine learning approaches are commonly employed in genetics and other areas where complex exposure patterns are indicated, such as exposome and neighborhood effects studies.^42–48 However, to our knowledge, only limited prior epidemiologic research has considered the potential relevance of such methods to policy analyses,^49–51 and no study has used these methods for opioid-related law analyses.

The aim of this study was to identify individual and prescription opioid-related law provision combinations predictive of high opioid dispensing and high-dose opioid dispensing in U.S. counties. We focused on these proximal outcome measures because many of the earliest policies directly targeted prescribing and dispensing behaviors. Moreover, as a large share of opioid prescriptions are written by a small number of providers, concentrated among a small share of patients, and escalating dose is associated with increased prescription opioid-related morbidity and mortality risk, high-dose dispensing is a frequent target of legislation.^52,53,54 Following recent work in the area of high-dimensional exposure modeling,^44,46,55 we used a multi-stage machine learning strategy combining regularization techniques and random forest algorithms to identify provisions that can be further tested in subsequent analyses with more causally oriented study designs. We also sought to explore overall patterns in policy configurations and their relationship to county-level dispensing over time.

METHODS

Setting

Because prescription opioid-related legislation has evolved considerably over time, we stratified analyses by overdose phase of the U.S. opioid epidemic: the prescription opioid phase (2006–2009), the heroin phase (2010–2012), and the fentanyl phase (2013–2016). While we focus on prescription opioid dispensing rather than overdose patterns, literature suggests that strategies to reduce inappropriate prescribing and dispensing have adapted contemporaneously with overdose trends.⁴ Stratification by period additionally helps improve balance in provision adoption across the timeframe (PDMP programs were implemented earlier, prescription opioid limits were implemented later), and further enables us to examine broader shifts in dispensing as law enactment patterns changed.

Outcomes

The primary outcomes were: (1) high county-level opioid dispensing, operationalized as counties within the highest quartile of dispensing per 100 persons per year, and (2) high-dose county-level opioid dispensing, operationalized as counties within the highest quartile of ≥90 daily morphine milligram equivalents (MME) dispensing, per 100 persons, per year. We collapsed county-level dispensing rates (highest quartile vs. others) within each time period to identify the counties most at-risk and for consistency with prior research.^56,57 Both dispensing measures were collected from the IQVIA longitudinal prescription database (2006–2016), a repository containing over 85% of U.S. retail pharmacy sales (additional details are in the online supplementary material).⁵⁸ Dispensing data for counties with ≤3 pharmacies is suppressed to prevent unmasking. County population estimates were from the U.S. Census Bureau.

Exposures

We restricted analyses to laws aimed at reducing hazardous prescribing and dispensing, including: PDMP, pain management clinic, and prescription opioid limit provisions. Other types of prescription opioid laws (e.g. overdose prevention laws etc.), and policies not formally codified as law (e.g., medical board regulations etc.), likely contribute to patterns in opioid-related harm. However, we focused on these domains because while interventions targeting provider behaviors have been shown to influence dispensing patterns and overdose deaths, prior studies have not examined provision heterogeneity and synergy across these domains systematically. Policies in included domains are more likely to shift behaviors immediately (i.e. within the first year of enactment), which is necessary requirement when comparing across policies systematically. PDMP and pain management clinic provision enactment dates were abstracted from the Prescription Drug Abuse Policy System (PDAPS).⁵⁹ Data on prescribing limit laws were obtained from Davis et al.⁶ Coding was based on the year each provision became operational. There was no limit on the months or years a provision was required to be effective to be included in the primary analyses. Table 1 includes examples of the 162 provisions examined; a comprehensive list is available in the online supplementary material (eTable 1).

Table 1.

Prescription opioid law provisions, 2006–2016

Category	Description	Dimensions	Examples of provisions	Number of provisions
Prescription Drug Monitoring Programs	Centralized statewide data systems that collect prescribing information when controlled substances are dispensed. Data may be accessed by authorized persons and regulatory agencies to facilitate appropriate prescribing and to identify potential sources of diversion.	Access and regulation features including which professions have access to the database and for what purpose, whether practitioners ca delegate their access, whether patients can view their own information, whether access can be granted for law enforcement purposes.	Law allows prescribers access to the PDMP data. Law allows prescribers to access PDMP data on current patients. Law allows in-state law enforcement to access PDMP data.	26
		Administration features including which department collects and stores data, how PDMP is funded, program powers and duties under the law.	Law permits the PDMP to report suspicious or statistically outlying prescribing, dispensing or purchasing activity to the prescriber or dispenser. Law requires evaluation of the PDMP. Law provide funding for the PDMP through charging fees.	29
		Reporting and authorization features including who is required to report to the PDMP, what drug schedules must be reported to the PDMP, whether the PDMP is authorized to share data with insurers, state Medicaid programs, or PDMPs located in other states.	Law requires federal drug schedule II to be reported to the PDMP. Law requires dispensers to check the PDMP before dispensing controlled substances to new patients. Law permits the PDMP to share data with other state PDMPs.	32
Pain management clinic laws	Laws that regulate pain management clinics, specific dimensions include: how PMCs are defined under the law, laws pertaining to clinic owners, laws pertaining to prescribers and other employees.		Law requires that a medical director be physically present at the pain clinic site. Law requires that pain clinic physicians cannot have prior felonies relating to prescription drugs or controlled substances. Law requires pain management clinics to conduct drug testing.	60
Initial prescribing limitations for prescription opioids	Laws that set restrictions on initial opioid prescriptions in terms of the specific controlled substances that are subject to regulation, duration, dose, and exceptions to limits.		Law limits initial prescription opioid limits to a 7 day supply or less. Law imposes additional prescription limits for minors. Law provides exception to initial prescribing limits for prescription opioids when patient is diagnosed with cancer.	15

Prescription Drug Monitoring Programs (PDMPs)

PDMPs are centralized statewide data systems that collect information when controlled substances are dispensed. Despite broad state adoption of PDMPs generally,^9,60 there is substantial heterogeneity in the specifics of these laws. We considered three classes of PDMP provisions: (1) access and use requirements (e.g. which provider types can access data), (2) administration (e.g. state agency responsible for oversight) and (3) reporting (e.g. types of drugs reported, Table 1).

Pain Management Clinic Laws

Pain management clinics regulate pain management clinics, often defined as health care facilities that are dedicated to or see a high number of patients for chronic pain diagnosis and management. The specific criteria for defining pain management clinics are typically a component of the state’s PMC law. Sometimes referred to as “pill mill” laws, these laws impose state oversight on pain clinics to reduce potentially risky prescribing. Examples of pain management clinic provisions include: restrictions on owners and prescriber qualifications, limits on amounts and types of medications prescribed, payment methods and requirements for routine inspections (Table 1).

Initial Prescription Opioid Prescribing Limits

Prescription opioid limits restrict the amount or duration of initial opioid prescriptions and are aimed at reducing the likelihood of patients developing unintended long-term opioid use and opioid use disorder. Prescribing limit provisions include: dosage limits on medications subject to regulation (type, drug schedule), duration limits (number of days’ supply), whether there are additional restrictions for minors, and exceptions to limits in special circumstances (e.g., surgical pain or palliative care, Table 1). We excluded other forms of dose restrictions not enacted directly by law, such as health department or professional guidelines or prior authorization policies.

Statistical Analysis

To identify the provisions and provision combinations most predictive of county-level, highest-quartile dispensing and high dose-dispensing, we applied a multi-stage machine learning method.⁶¹ First, we used basic dimension reduction (step 1) and least absolute shrinkage and selection operator (LASSO) regression⁶² (step 2) for pre-processing to reduce instability in final models. Next, we identified the specific provisions with the greatest predictive power using random forest models⁶¹ (step 3), then explored the extent to which combinations of provisions predicted dispensing (step 4). Although the optimal approach to building hypothesis-generating models has been debated, such multi-step strategies—using initial screening algorithms followed by final data modeling—may outperform single-step strategies by reducing noise from the inclusion of variables with small or null associations with the outcome.⁵⁵ Epidemiologic analyses examining other complex exposure patterns have successfully used this approach previously.⁴⁴

In step 1, we pre-processed the initial set of candidate provisions to address zero variance or complete collinearity within each provision group (PDMPs, pain management clinics, and prescription opioid limits) by period. Zero variance provisions were excluded from further analyses. We also combined related provisions (e.g., allowances for nurse practitioners and physicians assistants to dispense opioids) with complete collinearity (eTable 6–8). We then used a 0.8 correlation threshold to further reduce the provision set, as including highly correlated variables can reduce LASSO prediction accuracy and prior literature uses similar cut-offs.^63–67 For each provision pair with a correlation above the threshold, we included the provision with the smallest mean absolute correlation with the other provisions in the group.

In step 2, we used binomial LASSO regression to identify the maximally informative subset of provisions for each period. LASSO is a statistical approach that reduces data dimensionality by setting a constraint parameter—or penalty—for the size of the coefficients in the regression.⁶² The penalty shrinks estimates for less relevant coefficients to zero, thereby dropping minimally informative variables from the analysis. We used 10-fold cross validation, a process whereby the data is repeatedly split into training and test sets, to select the largest value of the penalty such that error is within 1 standard error of the minimum. Beyond the simple step 1 variable reduction approach, LASSO was used to further reduce dimensionality and multicollinearity prior to fitting the final models because prediction models with many low-relevance covariates can be unstable in terms of variable selection. Variables with nonzero coefficients following LASSO regression were retained in subsequent step 3 models.

We then used random forest models to identify provisions predictive of the highest dispensing and highest-dose dispensing counties (step 3), and to investigate provision interactions (step 4). Random forest is an ensemble of individual decision trees that can be used to best predict an outcome across randomly selected bootstrapped samples.^61,68 Underlying decision classification trees are built using recursive partitioning by repeatedly identifying splits in the data to identify the paths (i.e., branches) that best distinguish observations with the outcome of interest. We used a manual tuning grid to identify the optimal number of provisions randomly sampled as candidates at each split. Across periods, we grew 500 trees in each forest, after establishing that 500 trees was sufficient to produce relatively stable predictions while being relatively computationally efficient, on balance. Algorithmic tuning parameters and model accuracy were assessed with out-of-bag error⁶⁹ to preserve the maximum sample sizes across the three time periods. Importantly, random forest models provide an indication of explanatory power—variable importance—but not an interpretable direction of association; therefore, absence or presence of a law could have been considered predictive. Variable importance was assessed by mean decrease in accuracy (i.e., reduction in accuracy without that provision included), such that the higher the value, the more important the provision was in differentiating high dispensing and high-dose dispensing counties from other counties across the forest.

Finally, we also used random forest models to identify prevalent combinations of provisions (i.e. branches) across trees within the forest to explore provision interactions.⁷⁰ For this step, we constructed the random forest, then counted the frequency of provision combinations associated with each dispensing outcome. Because correlation between provisions can cause spurious identification we used a permutation test⁷¹ to identify the threshold above which a combination frequency would be suggestive of a predictive signal—in other words, the number of times a branch needed to appear in the forest to be considered inconsistent with random chance. Using this approach, the outcome is randomly permuted but correlation structures between provisions are maintained. Through bootstrapping, we created multiple forests using the permuted data, also generating a null distribution of combination frequencies. We used a conservative 99.9^th percentile threshold to identify the specific combinations most incompatible with this null distribution. Combinations represent potentially relevant interactions to be followed up in future, targeted analyses. Although branches theoretically represent combinations of provisions, branches containing only single provisions were possible if that provision alone was informative. We grew forests with 500 trees and permuted the outcome 1,000 times, limiting the candidate set to provisions previously identified with LASSO regression.

Robustness and sensitivity checks were conducted to: (1) address potential concerns about correlated observations across years by repeating analyses within single years of data; (2) test exposure coding assumptions by requiring that laws be enacted for ≥6 months to be considered in effect (3) directly evaluate predictive performance using a cross-validation (rather than out-of-bag estimation) on held-out test set of 20% of counties; and to explore the utility of the multistep process by (4) excluding the LASSO pre-processing steps, and (5) by including interaction terms directly in LASSO models (additional details are in the online supplementary material).

We excluded county–years for which no data were available (eTable 2). We conducted all analyses in R version 3.2.3. The Columbia University and New York University Institutional Review Boards approved study procedures.

RESULTS

Prescription opioid phase (2006–2009)

From 2006 to 2009, no states had enacted prescription opioid limits. After pre-processing, random forest models indicated that the most important provision predicting high dispensing in the prescription opioid phase was whether the PDMP law allowed dispensers to access prospective patient data (eFigure 1). Similarly, for high-dose dispensing, models identified prescriber and dispenser access to former patient data as highly discriminant, however, more of the top predictors were related to PDMP administrative features, especially funding-related provisions (Table 2, eFigure 1).

Table 2.

Machine learning results identifying prescription opioid law provisions predicting county-level high opioid prescribing and high-dose opioid prescribing, 2006–2016

		PDMP Access	PDMP Admin	PDMP Report	PMC	PO Limit	Total
	High Prescription Opioid Dispensing a
Prescription opioid overdose phase (2006–2009)	Step 1: Dimension reduction provisions retained	15	26	14	6		61
	Step 2: LASSO provisions retained	11	20	10	1		42
	Step 3: RF top 20 predictorsb	7	8	5	0		20
	Step 4: RF permutation	99.9% Permutation thresholdc: 15 Combinations identified above the threshold out of 1,988 combinations identified in total: 8
Heroin overdose phase (2010–2012)	Step 1: Dimension reduction provisions retained	17	27	20	20	0	84
	Step 2: LASSO provisions retained	12	11	14	9	0	50
	Step 3: RF top 20 predictorsb	6	5	5	4	0	20
	Step 4: RF permutation	99.9% Permutation thresholdc: 34 Combinations identified above the threshold out of 2,348 combinations identified in total: 0
Fentanyl overdose phase (2013–2016)	Step 1: Dimension reduction provisions retained	14	27	24	28	6	99
	Step 2: LASSO provisions retained	14	14	12	7	4	51
	Step 3: RF top 20 predictors	6	3	5	5	1	20
	Step 4: RF permutation	99.9% Permutation thresholdc: 3 Combinations identified above the threshold out of 3,054 combinations identified in total: 0
	High-Dose Prescription Opioid Dispensing a
Prescription opioid overdose phase (2006–2009)	Step 1: Dimension reduction provisions retained	20	28	28	29		105
	Step 2: LASSO provisions retained	8	13	3	2		26
	Step 3: RF top 20 predictorsb	4	12	3	1		20
	Step 4: RF permutation	99.9% Permutation thresholdc: 30 Combinations identified above the threshold out of 2,150 combinations identified in total: 5
Heroin overdose phase (2010–2012)	Step 1: Dimension reduction provisions retained	17	20	27	17	0	81
	Step 2: LASSO provisions retained	9	16	15	6	0	46
	Step 3: RF top 20 predictorsb	5	6	6	3	0	20
	Step 4: RF permutation	99.9% Permutation thresholdc: 32 Combinations identified above the threshold out of 2,990 combinations identified in total: 0
Fentanyl overdose phase (2013–2016)	Step 1: Dimension reduction provisions retained	14	27	23	56	3	123
	Step 2: LASSO provisions retainedb	9	21	21	30	1	82
	Step 3: RF top 20 predictors	6	5	3	6	0	20
	Step 4: RF permutation	99.9% Permutation thresholdc: 11 Combinations identified above the threshold out of 2,156 combinations identified in total: 1

LASSO: least absolute shrinkage and selection operator; RF: Random forest; PDMP: Prescription drug monitoring program; PMC: Pain management clinic; PO limit: Initial prescription opioid limit

^aHigh county-level opioid dispensing was operationalized as counties within the highest quartile of total opioid dispensing per 100 persons per year; high-dose county-level opioid dispensing, operationalized as counties within the highest quartile of ≥90 morphine milligram equivalents (MME) dispensing per day, per 100 persons, per year.

^bProvisions ranked by mean decrease in accuracy, a metric capturing the decrease in accuracy from permuting the out-of-bag values in each provision (i.e. the reduction in accuracy without that provision included), such that the higher the value, the more important the provision was in differentiating high dispensing and high-dose dispensing counties from other counties across the random forest.

^cThe permutation threshold was identified by randomly permuting the outcome variable 1,000 times and applying the random forest algorithm to create counts of null combinations. The threshold is then identified as the number of times above which a provision or provision combination frequency would need to appear in the random forest to be inconsistent with random chance at the 99.9th percentile (i.e. 0.001-level).

No initial prescription opioid limit provisions were in effect during the prescription opioid overdose phase.

Permutation analysis identified 8 combinations surpassing the 99.9^th percentile high dispensing threshold and 5 combinations surpassing the high-dose dispensing threshold—out of 1988 and 2150 randomly permuted combinations, respectively (Table 2). For both outcomes, each combination included at least one provision related to dispenser or prescriber PDMP patient data access (eTable 3). For high dispensing, 3 of 8 combinations also included provisions related to PDMP funding while for high-dose dispensing, 4 of 5 combinations also included law enforcement or delegate access features (eTable 4).

Heroin phase (2010–2012)

In the heroin phase, random forest results similarly identified prescriber and dispenser patient data access provisions as the first and second most predictive provisions for high-dispensing and the first and third most predictive provisions for high-dose dispensing (eFigure 2). In contrast to the prescription opioid phase however, several pain management clinics provisions also ranked as important, including the definition used to determine which types of facilities pain management clinic laws apply to, and pain management clinic-specific prescribing and dispensing limits (Table 2). For high-dose dispensing, the requirement that pain management clinic employees be licensed physicians was the second most important provision (eFigure 2).

From 2010 to 2012, permutation analysis did not identify any law provision combinations above the 99.9^th percentile threshold for either high dispensing (threshold: 30) or high-dose dispensing (threshold: 32) out of 2348 and 2990 combinations evaluated (eTables 3 and 4).

Fentanyl phase (2013–2016)

Between 2013 and 2016, the most important variable in the high dispensing random forest was the requirement that practices advertising pain services be subject to pain management clinic laws. Other highly ranked clinic variables included certification requirements and limits on opioids prescribed or dispensed in pain management clinics (eFigure 3). As in the earlier phases, prescriber and dispenser, prospective and former patient data access provisions also ranked highly for both outcomes. Pain management clinic provisions identified as important included requirements that employees be licensed physicians, and limits on opioids prescribed or dispensed in pain clinics.

For the fentanyl phase permutation results, 4 prevalent combinations were identified for high-dispensing (threshold: 9; total combinations: 3054) and 1 combination was identified for high-dose dispensing (threshold: 11; total combinations: 2156, Table 2, eTables 3 and 4). For high dispensing, identified combinations included pain management clinic requirements along with dispenser access to former patient data—for example criteria regarding the types of facilities that Pain management clinic laws apply to, disciplinary penalties for clinic law non-compliance, and requirements barring owners with prior felony convictions from operating clinics. For high-dose dispensing, the single combination identified was comprised of provisions stipulating dispenser access to former patient PDMP data and the requirement that dispensers report PDMP data weekly.

Sensitivity Analyses

In sensitivity analyses: (1) using single years, (2) requiring 6 months of enactment, (3) holding out a 20% test set, (4) excluding LASSO pre-processing and (5) including LASSO interaction terms, the results were broadly consistent with the primary analysis but some key differences were noted (eFigures 4–18). Overall, the most important variables selected by the random forest in sensitivity models were generally consistent with most important variables selected in the primary analysis, albeit in different orders. Across all models, however, predictive performance was low for both primary and sensitivity analyses, indicating that prescription opioid laws alone do not strongly predict dispensing (eTables 17–28). For the single-year and 6-month exposure coding, accuracy was qualitatively comparable with the primary analysis on balance; both sensitivity analyses had better error rates in the fentanyl period but worse sensitivity in identifying the highest-quartile dispensing and high-dose dispensing counties. Model performance for the held-out test tests were similar across analyses. In analyses including interaction terms directly within the LASSO, error rates were similar to the primary analysis but sensitivity was lower in models post-2010. Similarly, for high dispensing, excluding preprocessing slightly worsened error rates in the prescription opioid period but improved them in the fentanyl period, although sensitivity was reduced for models post-2010. For the high-dose dispensing, including interactions in the LASSO similarly increased performance for the fentanyl period but decreased sensitivity overall.

DISCUSSION

Between 2006 and 2016, PDMP provisions related to prescriber and dispenser patient data access most consistently predicted high dispensing and high-dose dispensing counties. For example, restricting to the five most important predictors across the three periods, PDMP patient data access provisions were included in approximately 53% of the top predictors for high dispensing (8 of 15); and 47% of the top predictors for high-dose dispensing (7 of 15)—out of 162 candidate law provisions included. PDMP patient data access provisions were also highly prevalent across provision combinations identified though random forest permutation analysis. Dispenser patient data access provisions were captured in 13 of 19 provision combinations identified; prescriber patient data access provisions were captured in 4 of 19.

In addition to the overall salience of PDMP patient data access, two key patterns emerged when comparing the predictive utility of various prescription opioid law provisions across phases of the U.S. opioid epidemic. First, pain management clinic-related provisions were not generally predictive of high dispensing or high-dose dispensing in the prescription opioid phase, likely because pain management clinic provisions tended to be adopted later than PDMP provisions. However, pain management clinic provisions became consistently more discriminant of high dispensing and high-dose dispensing counties over time, especially in the fentanyl period. These findings regarding the potentially important role of pain management clinic laws on prescribing behaviors post-2013 are in line with prior research indicating that these laws can contribute to reductions in inappropriate prescribing and dispensing.^5,18–25,30 We extend this work by suggesting some specific features of pain management clinic laws, particularly: which types of facilities the law applies to, clinic-specific opioid prescribing and dispensing limits, and clinic employee requirements, that might be tested in future analyses with more causally oriented study designs, to identify which types of pain management clinic laws may have the most benefit.

A second important pattern was that during the fentanyl period, provisions that in combination predicted high dispensing tended to be comprised of provisions across law domains. That is, most combinations identified above the permutation threshold included PDMP access provisions in combination with pain management clinic provisions (e.g., facility types subject to pain management clinic laws, disciplinary penalties, owner requirements). These provisions also became more prevalent in terms of variable importance rankings in the fentanyl period for both dispensing outcomes. Findings may point to the possibility of important synergistic relationships between laws in different prescription opioid domains, including the joint effects of PDMP patient data access and pain management clinic laws, and the relative importance of law provisions post-2013 overall, that deserve further study.

Limitations are noted. First, our study is exploratory by design, and used data-driven machine learning approaches to systematically identify prescription opioid law provisions that were associated with high dispensing and high-dose dispensing. To the extent that provisions are not independent, substantial further validation, especially research employing other types of study designs, are required to determine if any of the provisions or combinations identified as predictive, causally impact prescription opioid dispensing. Many factors other than legislation can strongly influence local dispensing patterns (e.g., demographic, socioeconomic, healthcare access, etc.), that we were unable to account for when identifying law provisions with the most predictive utility. Indeed, overall predictive performance across models was poor, suggesting that prescription opioid laws alone do not strongly predict dispensing. However, the goal of this study was to identify the most predictive provisions out of a large set of candidates to test in further research—as opposed to creating the most optimal prediction model—therefore we did not consider inclusion of other causes of opioid dispensing.

Second, although prescription opioid laws are implemented at the state-level, and are rarely reversed in subsequent years, we did not account for potential spatial or temporal dependencies. When we restricted the analyses to single years, results were broadly consistent with the primary analysis but potential spatial dependencies remained unaddressed. Future machine learning-based analyses might incorporate strategies to better address these dependencies to enhance hypothesis-generating prediction tasks. Future causally oriented analyses should account for such spatial and temporal clustering directly.

Additional methodologic concerns include the fact that while multi-step machine learning analyses are well-established, there is little consensus regarding which algorithms may perform best, and in which order. Other studies have used random forests for dimension reduction and LASSO for subsequent modeling.⁷² In contrast, we used LASSO models for pre-processing, as variable selection in random forest models may be unstable when low-relevance or correlated variables are included, and because we were interested in identifying provision combinations—a task better suited to random forest rather than LASSO approaches. Finally, because our analyses included all provisions simultaneously—so that we could systematically compare provisions in terms of predictive relevance—we could not incorporate time lags beyond the 6-month sensitivity analysis. Several of the law provisions we included may be associated with delayed effects, but we were unable to address differential latencies between policies directly.

In terms of data limitations, we did not consider policies not directly enacted as law (e.g., insurance policies, etc.), and we did not have access to other potentially relevant indicators of prescription opioid dispensing (e.g., overlapping prescriptions, concurrent sedatives, etc.). Future research incorporating these measures, and directly examining prescription opioid misuse, opioid use disorder, and overdose are warranted. Furthermore, there was a substantial degree of suppressed data, particularly for the high-dose dispensing outcome. Finally, our study did not capture data pre-2006 or post-2016 and thus, misses information from the first wave of the opioid epidemic and the potential impacts of more recent legislation.⁶

To our knowledge, this is the first paper to use machine learning to systematically identify high-relevance prescription opioid law provisions and provision combinations from a large set of candidates, to generate hypotheses about the types of opioid legislation that may deserve additional attention. Further research employing diverse study designs is needed to better understand how opioid laws generally, and these provisions specifically, can limit inappropriate opioid prescribing and dispensing to reduce opioid-related harms.

Figure 1.

Ranked importance of the top 20 predictors of high opioid dispensing and high-dose dispensing by mean decrease in accuracy in the prescription opioid phase, 2006–2009. Mean decrease in accuracy represents the decrease in accuracy from permuting the out-of-bag values in each provision (i.e. the reduction in accuracy without that provision included), such that the higher the value, the more important the provision was in differentiating high dispensing and high-dose dispensing counties from other counties across the random forest. Additional details are provided in the digital supplemental content.

DOH/HHS: Department of Health/Health and Human Services

LE: Law enforcement

P/D: Prescriber/dispenser

AG: Attorney General