Using routinely collected data to understand and predict adverse outcomes in opioid agonist treatment: Protocol for the Opioid Agonist Treatment Safety (OATS) Study

Abstract

Introduction

North America is amid an opioid use epidemic. Opioid agonist treatment (OAT) effectively reduces extramedical opioid use and related harms. As with all pharmacological treatments, there are risks associated with OAT, including fatal overdose. There is a need to better understand risk for adverse outcomes during and after OAT, and for innovative approaches to identifying people at greatest risk of adverse outcomes. The Opioid Agonist Treatment and Safety study aims to address these questions so as to inform the expansion of OAT in the USA.

Methods and analysis

This is a retrospective cohort study using linked, routinely collected health data for all people seeking OAT in New South Wales, Australia, between 2001 and 2017. Linked data include hospitalisation, emergency department presentation, mental health diagnoses, incarceration and mortality. We will use standard regression techniques to model the magnitude and risk factors for adverse outcomes (eg, mortality, unplanned hospitalisation and emergency department presentation, and unplanned treatment cessation) during and after OAT, and machine learning approaches to develop a risk-prediction model.

Ethics and dissemination

This study has been approved by the Population and Health Services Research Ethics Committee (2018HRE0205). Results will be reported in accordance with the REporting of studies Conducted using Observational Routinely-collected health Data statement.

Strengths and limitations of this study.

• The use of a population cohort of people with opioid use disorder, and people moving in and out of opioid agonist treatment with methadone and/or buprenorphine over an extended period (2001–2017).

• Linkage of disparate datasets addressing physical health, mental health and substance use, criminal justice and mortality.

• Cross-national funding and collaboration to inform responses to an epidemic.

• A key limitation is a lack of primary care data to better quantify physical comorbidity.

Introduction

There have been dramatic increases in opioid-related morbidity and mortality across many high-income countries, including the USA, Canada and Australia,^1–3 driven by increases in pharmaceutical opioid use,^4–6 and in North America, a rapid rise in the availability of illicit heroin and fentanyl.^{7 8} In the USA, there are now over 100 opioid overdose deaths per day.¹ Multiple HIV and hepatitis C outbreaks linked to opioid injection have occurred, often in rural areas, illustrating the reach of the crisis.^9–13 The Council of Economic Advisers put the impact at US$504 billion in 2015, nearly 3% of gross domestic product.¹⁴

As a chronic relapsing disease, access to effective treatment for opioid use disorders is a critical component of the response to the opioid epidemic.⁴ Opioid agonist treatment (OAT) is the most effective treatment for opioid use disorder,¹⁵ including prescription opioid use disorder.¹⁶ The main medications used for OAT are methadone, a full opioid agonist, and buprenorphine, a partial opioid agonist.

Risk of death from any cause is halved while on OAT compared with out-of-treatment opioid use disorder; risk of overdose death is reduced by 70%.¹⁷ Other harms from opioid use and injecting are also reduced while on OAT. HIV and hepatitis C acquisition are halved^{18 19} while on treatment. Health service use, particularly unplanned and crisis-driven hospitalisations and emergency department presentations, are significantly reduced.^20–22 Criminal offending and associated costs are significantly reduced.^23–25

As with any pharmacological treatment, there are risks associated with OAT. Periods of elevated risk of fatal overdose are observed during treatment induction (the first 4 weeks), particularly with methadone, and immediately following treatment cessation.^26–28 Additional adverse outcomes during OAT may include non-fatal overdoses, accidents²⁹ or cardiac problems resulting from QT interval prolongation.³⁰ Adverse outcomes during OAT may also include events that are not directly related to treatment, but are still of importance due to their potential severity and impact on clinical care; for example, instances of self-harm and suicide. To date however, much of the postmarketing surveillance of methadone and buprenorphine has focused on diversion and illicit use of medications,^31–33 with relatively less attention given to adverse outcomes of people in treatment or following treatment cessation. This is a significant knowledge gap that leaves OAT recipients at risk of harm.

Many questions in relation to adverse outcomes in OAT remain unanswered. Most importantly, it is not clear what patient, provider and treatment setting factors may influence mortality risk. There are also less severe, but likely more common, adverse outcomes in OAT that have largely been unexamined. For example, how common are non-fatal overdoses or other injuries during OAT induction? Are these outcomes more common among particular groups of people or in particular treatment settings? What other adverse outcomes occur after leaving treatment?

Identifying OAT recipients at highest risk of adverse outcomes

In the USA, OAT is offered in both clinic (where methadone and buprenorphine may be prescribed) and office-based (buprenorphine prescribing only) settings which provide varying levels of monitoring and support. There are a variety of factors that play into matching patients with treatment setting, including the level of training of the provider (generalist vs specialist) and supportive resources (on-site vs off-site), and to date there are no evidence-based algorithms to guide care.

Information regarding substance use history, previous treatment, comorbidities, social supports and other factors that may influence treatment planning is obtained from people seeking OAT via clinical assessment.^34–36 However, individuals may be reluctant at an initial assessment to disclose details about the extent of their substance use and related issues, such as disconnection from family and criminal justice matters, or about mental illness and suicidality. Furthermore, there is no quantification of how much weight to give any specific issue, or combination of issues, in deciding on the most appropriate and safest treatment strategy for a specific individual. Although treatment guidelines emphasise the importance of identifying the appropriate treatment setting for people with risk factors for poorer outcomes,^{35 36} they do not consider more complex provider and setting factors, such as provider training, experience and caseload. Given the range of clinical, sociodemographic and treatment setting/provider factors that are likely to affect risk in OAT, and the possibility that these factors may interact in currently unknown ways to magnify (or reduce) risk, self-reported histories alone are unlikely to provide sufficient detail to understand and predict individual risk.

Consistent with a move towards personalised medicine, there is a need for innovative approaches to identifying people at highest risk of adverse outcomes in OAT, that take into account medication-specific risks, the individual’s history and provider factors, to produce an individualised probability of adverse outcome that can be used to guide treatment strategy. Machine learning offers such an approach. Machine learning is a flexible tool that allows for exploration of a much wider range of potential predictors and combinations of predictors than standard regression models. While standard regression models are focused on determining the direction and magnitude of risk conferred by specific factors, machine learning aims to maximise the predictive ability of a model. This approach has previously been used to identify Army veterans most at risk of suicide.³⁷

There is considerable potential for machine learning to predict the probability of a specified adverse outcome in OAT through real-time application of an algorithm to routinely collected data. In practice, this will allow for the development of individualised treatment plans that take into account the individual’s specific probability of specific adverse outcomes, minimising risk and maximising treatment retention.

To address the need for greater knowledge of the frequency and magnitude of risk for adverse outcomes during and after OAT to support the expansion of OAT in the USA, the Opioid Agonist Treatment and Safety Study aims to:

1. Determine the magnitude of risk for specific adverse outcomes (eg, mortality, unplanned hospitalisation and emergency department presentation, and unplanned treatment cessation) during and after OAT with methadone and buprenorphine.

2. Identify patient (sociodemographic characteristics, comorbidities), treatment (clinic or office-based setting, starting dose) and provider (training, caseload) factors associated with adverse outcomes during and after OAT with methadone and buprenorphine.

3. Develop risk-prediction models to identify patients at greatest risk of adverse outcomes during and after OAT.

Methods and analysis

Design

This is a retrospective cohort study using linked, routinely collected health and criminal justice data. The cohort includes all people seeking OAT in New South Wales (NSW), Australia, between 2001 (when buprenorphine was approved for use for the treatment of opioid dependence in Australia) and 2017, with linkage to hospital admissions, emergency department presentations, mental health diagnoses, incarceration episodes and mortality.

Setting

Approximately 40% of Australia’s OAT recipients reside in NSW.³⁸ Methadone and buprenorphine (including buprenorphine–naloxone) may be prescribed by physicians or nurse practitioners. OAT is dispensed in a variety of settings including public and private clinics, community pharmacies and correctional facilities. In rural areas, local hospitals may also dispense OAT medicines. All OAT is provided as daily supervised doses for the first 3 months of treatment. After 3 months, take-home doses may be provided at the discretion of the prescribing doctor, with the number of take-home doses per week increasing with duration in treatment. The maximum recommended number of take-home methadone doses is four. OAT patients prescribed buprenorphine–naloxone may progress to a 28-day prescription of dispensed medicine if they have demonstrated stability on a 2-week prescription over a number of months, but this rarely occurs in clinical practice. There are no charges for OAT recipients treated in public clinics or correctional facilities; OAT recipients who have their medication dispensed at private clinics or community pharmacies are charged daily dispensing fees (typically $A5–$A8 per day).^{39 40}

Datasets

The primary database for this linkage is the Electronic Reporting and Recording of Controlled Drugs (ERRCD) system. The ERRCD (formerly Pharmaceutical Drugs of Addiction System) contains records of all OAT prescribed in NSW, in any setting. People receiving OAT between 2001 (when buprenorphine was first approved for use in OAT in NSW) and 2017 form the cohort for this study. Based on previous linkages, we estimate that the cohort will include at least 45 000 unique OAT recipients and over 600 000 person-years of observation.

As people seeking OAT must provide identification documents to obtain a prescription, personal identifiers in the ERRCD are considered reliable for probabilistic linkage to other routinely collected data. For this study, the ERRCD is probabilistically linked to five state-wide databases; linked variables are shown in table 1. The linkage process is managed by the Centre for Health Record Linkage in collaboration with data custodians, using best practice privacy-preserving protocols.⁴¹ Researchers do not receive identifying data for any individual in the cohort at any time.

Table 1.

Databases linked in the Opioid Agonist Treatment and Safety Study

Database name	Database description	Linked variables*
Electronic Reporting and Recording of Controlled Drugs	Authorisation for dispensing methadone or buprenorphine for the treatment of opioid dependence.	Dates of OAT entry and cessation. Primary opioid of concern. Start and final dose. Prescriber identification number. Dosing point. Date of provider accreditation. Date of provider’s first OAT authority. Statistical area of OAT recipient’s address. Statistical area of OAT provider’s practice. Reason for OAT cessation.
Admitted Patients Data Collection	All hospitalisations in all public, private, psychiatric and repatriation hospitals in NSW.	Dates of admission and separation. Planned or unplanned admission. Diagnoses (underlying and contributing). Procedures. Mode of separation.
Emergency Department Data Collection	Presentations to emergency departments in NSW.	Date of presentation and separation. Triage category. Diagnosis (underlying only). Planned or unplanned presentation. Mode of separation.
Mental Health Ambulatory Data Collection	Mental healthcare for non-admitted patients, including day programmes, psychiatric outpatients and outreach services.	Mental health diagnoses (primary and additional).
Re-offending Database	Court appearances, juvenile detention and adult incarceration in NSW.	Dates of prison reception and release. Level of Service Inventory-Revised risk category.
Registry of Births, Deaths and Marriages and Cause of Death Unit Record File	Deaths registered in NSW.	Date of death. Cause of death (underlying and contributing).

*All linked databases include sex, month and year of birth, and Indigenous status.

NSW, New South Wales; OAT, opioid agonist treatment.

Outcomes

Adverse outcomes that will be assessed as outcomes are all-cause, drug-induced, suicide and injury deaths, as well as unplanned hospitalisations and emergency department presentations related to each of these categories. International Classification of Diseases 10th Revision codes for defining these outcomes are shown in table 2. Outcomes will be measured during four distinct time periods during and out of OAT (treatment-by-time periods):

• Induction: the first 28 days of a new episode of OAT.

• Remainder of time in OAT: the 29th day of OAT onwards, until the end date of the treatment episode or end of follow-up.

• Immediately post-OAT: the first 28 days after the end of a treatment episode.

• Remainder of time out of OAT: the 29th^h day following OAT cessation onwards, until the first day of a new treatment episode, or end of follow-up.

Table 2.

Defining ICD-10 codes for adverse outcomes during and after OAT

Outcome	ICD-10 codes
Drug-induced	F11-F16, F19, F55, X40-X44, X60-X64, X85, Y10-Y14
Suicide/self-harm	X60-X84, Y87.0
Injury	V01-X59

ICD-10, International Classification of Diseases 10th Revision; OAT, opioid agonist treatment.

Statistical analysis plan

To address aims 1 and 2 (determining the magnitude and risk factors for adverse outcomes during and after OAT), we will allocate all adverse outcomes to the specified treatment-by-time periods. We will calculate rates of adverse outcomes using standard methods and 95% Poisson CIs. Poisson regression will be used to calculate adjusted incidence rate ratios for specific adverse outcomes, comparing rates during/after methadone to the same time periods for buprenorphine. Should concerns arise regarding confounding by indication, we will explore methods such as propensity score matching and instrumental variables (assuming we can identify a reliable indicator of the probability of a clinician prescribing buprenorphine). In these analyses, special attention will be given to testing whether patients who transfer from buprenorphine to methadone during a treatment episode are at increased risk of death during and after OAT, and whether unplanned treatment cessation is reduced, compared with patients who remain on buprenorphine.

We will use survival analysis methods to identify risk factors—including patient, setting and provider factors—for adverse outcomes during specific treatment-by-timetime periods. Each adverse outcome specified above will be considered separately. We will investigate the effect of repeated/interrupted treatment exposure by using methods that incorporate multiple observations per person (eg, frailty models; generalised linear mixed models or generalised estimating equations). Risk factors will be incorporated into multivariable models on the basis of results of univariate analyses. Risk factors will be separately identified for methadone and buprenorphine, during specific treatment-by-time periods. As exact dates of incarceration are included in the linked dataset, we will account for time in custody as a time-dependent covariate, and examine how risk of adverse outcomes varies by incarceration status.

Aim 3 is to develop risk-prediction models to identify patients at greatest risk of adverse outcomes during and after OAT. We will use two main approaches to machine learning to develop our risk-prediction models: tree-based methods and methods based on logistic regression. Classification trees identify individuals at increased risk of adverse outcomes by stratifying or segmenting the population into groups that maximise the predictive accuracy. This approach differs from conventional regression analysis in that no assumptions are required as to the nature of the relationship between the predictors and outcomes. In particular, it does not assume linearity or additivity of predictor variables. We will use random forests, a machine-learning method, for classification⁴² that generates many classification trees and ranks their predictive importance. Each tree is based on a separate bootstrapped pseudo-sample of the original dataset which protects against over-fitting.⁴²

Logistic penalised regression models will be developed.⁴³ This approach ‘penalises’ what one would consider less realistic values of the unknown model parameters, reducing instability caused by high correlations among predictors. Three penalties will be evaluated: the ridge penalty, the lasso penalty and an intermediate mixing parameter penalty. The ridge penalty uses proportional coefficient shrinkage to retain all predictors. The lasso penalty favours sparse models that force coefficients for all but one predictor in each strongly correlated set to zero. The intermediate elastic net penalty combines both approaches.⁴⁴

We will compare the candidate models from each technique and create additional model ensembles which combine prediction outcomes across candidate models and can produce superior models. Models will be trained using k-fold cross-validation and the champion model will be selected based on a comparison of the area under the curve of the original algorithms and the ensemble models.

Patient and public involvement

There was no patient or public involvement in the development of the study design. A Community Reference Panel was consulted as part of the process of gaining ethical approval for the study; this involved consultations with people who inject drugs and people with experience of OAT, including representation of Aboriginal people with OAT experience. These consultations provided information to inform our approach to disseminating findings of this project. We will prepare one-page summaries of key findings for distribution to OAT clinics, other drug treatment services and harm-reduction services.

Ethics and dissemination

Data storage, retention and access

To protect privacy and confidentiality, approval for the linkage of health data in NSW is provided under strict conditions for the storage, retention and use of the data. The current approval permits storage of the data at one site, the University of New South Wales, for up to 7 years following the date of publication of results. Researchers wishing to undertake additional analyses of the data are invited to contact the corresponding author to discuss requirements, including the need to obtain approval from the Population and Health Services Research Ethics Committee for data access. Data may only be supplied for analysis within Australia.

Dissemination

A key audience for dissemination will be US-based addiction medicine and addiction psychiatry physicians and policy stakeholders. Project findings will be disseminated at scientific conferences and in peer-reviewed journals, with policy briefs distributed via a website. Outputs will include publications examining the magnitude of and risk factors for adverse outcomes during and after OAT, the development and performance of the machine-learning model, and methodological papers. As the study uses routinely collected health data, findings will be reported in accordance with the REporting of studies Conducted using Observational Routinely-collected health Data statement.⁴⁵