Skip to main content
The BMJ logoLink to The BMJ
. 2020 Mar 4;368:m283. doi: 10.1136/bmj.m283

Associations between stopping prescriptions for opioids, length of opioid treatment, and overdose or suicide deaths in US veterans: observational evaluation

Elizabeth M Oliva 1 ,2,, Thomas Bowe 1 ,2, Ajay Manhapra 3 ,4 ,5 ,6, Stefan Kertesz 7 ,8, Jennifer M Hah 9, Patricia Henderson 1, Amy Robinson 10, Meenah Paik 1, Friedhelm Sandbrink 11 ,12 ,13, Adam J Gordon 14 ,15 ,16, Jodie A Trafton 1 ,2 ,17
PMCID: PMC7249243  PMID: 32131996

Abstract

Objective

To examine the associations between stopping treatment with opioids, length of treatment, and death from overdose or suicide in the Veterans Health Administration.

Design

Observational evaluation.

Setting

Veterans Health Administration.

Participants

1 394 102 patients in the Veterans Health Administration with an outpatient prescription for an opioid analgesic from fiscal year 2013 to the end of fiscal year 2014 (1 October 2012 to 30 September 2014).

Main outcome measures

A multivariable Cox non-proportional hazards regression model examined death from overdose or suicide, with the interaction of time varying opioid cessation by length of treatment (≤30, 31-90, 91-400, and >400 days) as the main covariates. Stopping treatment with opioids was measured as the time when a patient was estimated to have no prescription for opioids, up to the end of the next fiscal year (2014) or the patient’s death.

Results

2887 deaths from overdose or suicide were found. The incidence of stopping opioid treatment was 57.4% (n=799 668) overall, and based on length of opioid treatment was 32.0% (≤30 days), 8.7% (31-90 days), 22.7% (91-400 days), and 36.6% (>400 days). The interaction between stopping treatment with opioids and length of treatment was significant (P<0.001); stopping treatment was associated with an increased risk of death from overdose or suicide regardless of the length of treatment, with the risk increasing the longer patients were treated. Hazard ratios for patients who stopped opioid treatment (with reference values for all other covariates) were 1.67 (≤30 days), 2.80 (31-90 days), 3.95 (91-400 days), and 6.77 (>400 days). Descriptive life table data suggested that death rates for overdose or suicide increased immediately after starting or stopping treatment with opioids, with the incidence decreasing over about three to 12 months.

Conclusions

Patients were at greater risk of death from overdose or suicide after stopping opioid treatment, with an increase in the risk the longer patients had been treated before stopping. Descriptive data suggested that starting treatment with opioids was also a risk period. Strategies to mitigate the risk in these periods are not currently a focus of guidelines for long term use of opioids. The associations observed cannot be assumed to be causal; the context in which opioid prescriptions were started and stopped might contribute to risk and was not investigated. Safer prescribing of opioids should take a broader view on patient safety and mitigate the risk from the patient’s perspective. Factors to address are those that place patients at risk for overdose or suicide after beginning and stopping opioid treatment, especially in the first three months.

Introduction

In the United States, concerns about the opioid crisis—the rise in deaths from overdose1 and opioid use disorders2 3 4—has prompted a range of initiatives to mitigate the risks associated with prescribing opioids for chronic pain, and to encourage tapering or stopping when risks exceed benefits.5 6 7 8 9 10 11 12 13 These efforts have seen a fall in the prescribing of opioids by 35% between 2012 and 2017, per 100 000 people.14 Guidance on reducing opioids typically warns against sudden changes to care. For example, a Veterans Affairs/Department of Defense Guideline specifies that “[a]brupt discontinuation should be avoided unless required for immediate safety concerns.”13 Some studies suggest a rise in adverse events after stopping treatment with opioids, but national studies with large sample sizes are lacking.

In 2012, a retrospective study was conducted in the Veterans Health Administration of 509 veterans whose long term treatment with opioids was stopped. Patients with substance use disorders were matched to patients with no disorders. The researchers found evidence of suicidal thoughts and self-harm in patients with and without substance use disorders who ended treatment with opioids after long term use.15 A nested case-control study from Kaiser Permanente Colorado in patients on long term treatment with opioids between 2006 and 2017 (228 patients who took an overdose of opioids matched to 3547 control patients) found that variations in the dose of opioid increased the likelihood of overdose. Stopping treatment for at least three months (that is, 0 mg of morphine equivalence) mitigated the risk.16 In an analysis of Vermont Medicaid data between 2013 and 2017 of 494 patients treated with high doses of opioids for long periods of time who stopped taking opioids, researchers found that almost half had been admitted to hospital or visited the emergency department for an opioid or substance use related event, after they had stopped treatment.17 The impact of stopping treatment in 572 patients on long term treatment with opioids was examined in a safety net primary care clinic in Seattle with opioid registry data from 2010 to 2015. Stopping opioid treatment was associated with an increased risk of death from overdose18 but the analyses could not assess the characteristics of the patients that might have contributed to the deaths. These characteristics could be important because patients with substance use disorders and mental health diagnoses have historically been more likely to be treated with opioids for chronic pain,19 which might increase the risk for psychiatric or medical instability when treatment with opioids is stopped.15 20 21 Given these uncertainties, Frank and colleagues called for research on the impact of reducing the prescribing of opioids.22

The Veterans Health Administration has taken a multifaceted approach to the safety of prescribing opioids. Sequential safety initiatives produced a 37% reduction in the prescribing of opioids between 2014 and 2016, largely by reducing the introduction of long term use of opioids, but also by reducing opioid treatment in some long term users.23 24 25 The purpose of our evaluation was to examine the likelihood of death from overdose or suicide in veterans prescribed opioid analgesics in the early implementation period of the Veterans Health Administration’s opioid safety initiative.24 We studied the interaction between stopping opioid treatment and length of opioid use because opioid tolerance, dependence, and addictive behaviors might develop with longer exposure to opioids and might contribute to the risk of death from overdose or suicide after treatment with opioids is stopped.21,26 27 28

Methods

Stanford University’s institutional review board determined that this quality improvement project was not research.

Patient population

The Veterans Health Administration’s Corporate Data Warehouse (extracts from electronic medical record data) identified patients with an outpatient prescription for an opioid analgesic in fiscal year 2013 (1 October 2012 to 30 September 2013). Opioids used to treat opioid use disorders were excluded (eg, sublingual buprenorphine formulations). Opioid prescriptions were for short acting or long acting drugs. Short acting opioids were butorphanol, codeine, dihydrocodeine, hydrocodone, hydromorphone, meperidine, morphine, opium/belladonna, oxycodone, oxymorphone, pentazocine, tapentadol, and tramadol. Long acting opioids were fentanyl transdermal patch, buprenorphine transdermal patch, methadone (excluding methadone from licensed opioid treatment programs), sustained release hydromorphone, sustained release morphine, sustained release oxycodone, sustained release oxymorphone, sustained release tapentadol, and sustained release tramadol.

Evaluation period

We examined the last opioid prescription episode in fiscal year 2013 and death related to overdose or suicide to the end of fiscal year 2014. Thus the evaluation period was 1 October 2012 to 30 September 2014 (fiscal year 2013 to fiscal year 2014).

Measures

Prescribing data for opioids and personal patient data for fiscal year 2013 (age, sex, current marital status, rural residence), number of Elixhauser medical diagnoses,29 and diagnoses for mental health and substance use disorders were from the Veterans Health Administration’s Corporate Data Warehouse.21 Prescribing data for opioids were used to identify a patient’s last opioid prescription episode in fiscal year 2013, maximum daily morphine milligram equivalents for an opioid prescription, type of opioid treatment, date of stopping treatment, and status of treatment (stopped or continued).

Last opioid prescription episode in fiscal year 2013

Opioid prescription episodes were identified by the drug name, quantity, days’ supply, and release dates for each patient with an opioid prescription from 1 October 2012 to 30 September 2014. We identified the last opioid prescription episode in fiscal year 2013 for every patient. To identify the start and end dates of an opioid prescription episode, prescription data were tracked for five years before the date of death or the end of the next fiscal year. An opioid prescription episode was defined as a period in which gaps in possession of opioids never exceeded twice the days’ supply for a newly written prescription, 180 days between releases of a refill of an existing prescription, or 300 days between any two prescription releases regardless of refill status.

Length of last opioid prescription episode in fiscal year 2013

The length of the last opioid prescription episode in fiscal year 2013 was calculated with the start and end dates of the last prescription for opioids (categorized as ≤30, 31-90, 91-400, and >400 days). This variable is described as length of treatment with opioids.

Maximum daily morphine milligram equivalents

Morphine equivalents for each patient’s opioid prescription(s) were calculated for fiscal year 2013 with the method described by the Centers for Disease Control and Prevention for calculating the daily amount of opioid prescriptions—that is, converting each opioid prescription into daily morphine milligram equivalents and adding them together.30 Daily morphine milligram equivalents for each patient’s Veterans Health Administration opioid prescription(s) were added together to give the maximum daily morphine milligram equivalents. The statistical analyses used log maximum daily morphine milligram equivalents.

Type of opioid treatment

Type of treatment was the opioid prescribed in fiscal year 2013 (hierarchical classification): long acting opioid, short acting opioid, or tramadol only.31 32

Date of stopping treatment and opioid status

The date when treatment with opioids was stopped was estimated for each patient as the date of the last prescription for opioids in addition to the days’ supply. The date when opioid treatment was stoppedwas when patients were estimated to have no pills, if that occurred before the date of death or before the end of fiscal year 2014. For descriptive use only, the date when treatment with opioids was stopped was used to create a binary status variable: stopped or continued treatment with opioids. Patients were assigned to a stopped or continued treatment group. Patients in the continuing treatment group were estimated to have pills on hand at the time of death or at the end of fiscal year 2014.

Outcome measure

Our previous work with this population found similar risk factors for overdose and suicide,21 and thus the primary outcome in this evaluation was a binary variable of death caused by overdose (including opioid, sedative, acetaminophen, or other drug poisoning) or suicide, according to ICD-10 (international classification of diseases, 10th revision), 10th revision, from the National Death Index search results of the Veterans Affairs/Department of Defense Mortality Data Repository (appendix 1). Deaths from overdose and suicide were not limited to those involving opioids.

The Veterans Affairs and Department of Defense jointly search the National Death Index for data on numbers and causes of death as part of their surveillance on suicide. Searches involve matches with death certificates in the National Death Index, based on name, social security number, date of birth, sex, race, and marital status. The National Death Index matches records in a two step process. The first step is selection of potential matches based on seven matching criteria: social security number; exact month and within a year of birth, and first and last name; exact month and within a year of birth, first and middle initials, and last name; exact month and day of birth, and first and last name; exact month and day of birth, first and middle initials, and last name; exact month and year of birth, first name, and father’s surname; and if the patient is a woman, exact month and year of birth, first name, last name (on user’s record), and father’s surname (on National Death Index record). A scoring and classification procedure results in a probabilistic score and suggested final match by the National Death Index. A possible match is a record that matches one of the seven criteria. The Veterans Affairs uses algorithms to find the best of all possible matches. Possible matches are run through several sets of potential true match criteria. A match is counted if a possible match meets one of the sets of criteria. Matches are run through additional criteria to determine a best match for each social security number by ranking the matching variables of the National Death Index by order of importance. The record with the highest priority matching criteria is selected for each social security number.

Cox non-proportional hazards regression model

We used SAS PHREG (UNIX SAS version 9.2; Executive Information Systems, Bethesda, MD) to conduct a Cox non-proportional hazards regression model that analyzed the last opioid prescription episode started in fiscal year 2013, with death from overdose or suicide as the primary outcome. The main covariate of interest was the interaction of time varying opioid cessation (date of stopping treatment) by length of treatment with opioids (≤30, 31-90, 91-400, and >400 days). SAS PHREG can test and correct for violations of proportional hazards assumptions inherent in the proposed model that includes an interaction with a function of time (length of treatment with opioids).33 34 Consistent with approaches that incorporate time related variables, length of treatment with opioids was only included in the interaction term with stopping opioid treatment. We also included stopping opioid treatment as a time varying covariate to help avoid immortal time bias.35 36 Also, to account for potential bias introduced by opioid treatment before the start of the observation period (that is, the index fiscal year), entry into the model was left censored based on length of opioid use before fiscal year 2013. For example, patients with an opioid prescription episode 30 days before the start of fiscal year 2013 were not included in the risk pool for the first 30 days of their treatment because patients could not have died from suicide or an overdose before the start of fiscal year 2013 and still enter the study. Thus patients were not included in the model until they entered the outcome observation period, with the index date for the model being the start of fiscal year 2013 (1 October 2012) for patients treated with opioids before fiscal year 2013 or the start date of the last opioid prescription episode that occurred in fiscal year 2013 for patients who were not treated with opioids before fiscal year 2013. Also, all models were adjusted for baseline demographic characteristics; number of medical diagnoses; diagnosis of a mental health, substance use, or nicotine use disorder; type of opioid treatment; and the log of the maximum daily morphine milligram equivalents. The analyses excluded individuals with incomplete data. Patients who died from causes other than overdose or suicide were right censored at the time of death. Cox non-proportional hazards regression models were also conducted for overdose and suicide outcomes separately as sensitivity analyses.

Life table descriptive data

We used SAS LIFETEST to better understand the potential clinical impact of the interaction between stopping treatment with opioids and length of treatment on death from overdose or suicide over time. With SAS LIFETEST, we created life tables that characterized observed death rates for overdose and suicide calculated every 25 days. Two life tables were created, the first characterized death from overdose or suicide in patients while treated with opioids and after stopping treatment, and the second characterized death from overdose or suicide after stopping treatment with opioids by length of treatment. For the first life table, death rates for overdose or suicide for patients who started an opioid prescription in fiscal year 2013 (to address immortal time bias for patients who started treatment before fiscal year 2013) were calculated from the start date of their last opioid prescription, when treated with opioids. For patients in the group who stopped treatment with opioids, death rates were calculated from the date of stopping treatment because, by definition, patients had to have survived up to the date of stopping treatment to enter this group. The first life table essentially shows death rates for overdose and suicide in patients who continued to be treated with opioids by length of treatment. The second life table was created to show death rates in patients who stopped treatment with opioids by length of opioid treatment. Appendix 2 includes a heuristic summary of the data assembly method used in this evaluation.

Patient and public involvement

Although patients and the public were not directly involved in this evaluation, the questions were informed by the concerns of patients and the public, as expressed directly through congressional representatives or the media to the Veterans Health Administration, the experiences of authors involved in this evaluation, and the concerns about potential iatrogenic effects of stopping treatment with opioids. As this was a quality improvement project, the findings have been shared internally on numerous national calls of the Veterans Health Administration with leadership and providers. A national workgroup was convened to identify additional quality improvement efforts to address findings from this evaluation (eg, medication use evaluation, natural language processing approaches). The Veterans Health Administration computerized decision support systems have been updated to highlight risks and facilitate monitoring and risk management of patients after stopping treatment with opioids.

Results

The number of patients with an outpatient prescription for an opioid analgesic in fiscal year 2013 was 1 394 102. Table 1 describes the characteristics of the patients based on treatment with opioids for fiscal year 2013. The incidence of stopping treatment with opioids was 57.4%. Mean age of the patients was around 60 years, about 8% were women, around half were married, and most (61%) lived in urban areas. Just over half had three or more medical diagnoses. Alcohol use disorders were diagnosed in about 10% of patients and nicotine use disorders in about 22%. The most common mental health conditions were depressive disorders and post-traumatic stress disorder.

Table 1.

Characteristics of patients by status of opioid treatment in fiscal year 2013

Characteristics of patients Stopped opioid treatment Continued opioid treatment
No (%) of patients 799 668 (57.4) 594 434 (42.6)
Mean (SD) age (years) 59.8 (15.4; 5 missing) 61.3 (12.6; 6 missing)
No (%) of women 70 299 (8.8) 42 691 (7.2)
No (%) currently married 384 211 (48.0) 300 073 (50.5)
Residence location:
 No (%) in urban areas 515 452 (64.5) 344 722 (58.0)
 No (%) in rural areas 284 216 (35.5) 249 712 (42.0)
Mean (SD) No of medical diagnoses 2.9 (2.3) 3.1 (2.2)
No (%) of patients with 0 to ≥5 medical diagnoses:
 0 109 958 (13.8) 51 935 (8.7)
 1 139 174 (17.4) 97 395 (16.4)
 2 149 358 (18.7) 116 378 (19.6)
 3 130 677 (16.3) 106 613 (17.9)
 4 98 644 (12.3) 81 674 (13.7)
 ≥5 171 857 (21.5) 140 439 (23.6)
No (%) diagnosed with substance use disorders: 116 830 (14.6) 82 006 (13.8)
 Nicotine 165 993 (20.8) 145 973 (24.6)
 Alcohol 85 205 (10.7) 56 322 (9.5)
 Drug 66 282 (8.3) 46 102 (7.8)
 Cannabis 25 467 (3.2) 14 697 (2.5)
 Opioid 16 213 (2.0) 13 177 (2.2)
 Barbiturate 3,276 (0.4) 2,112 (0.4)
No (%) of patients with mental health diagnoses: 343 478 (42.9) 289 636 (48.7)
 Affective disorders 29 049 (3.6) 22 898 (3.8)
 Schizophrenia 15 637 (2.0) 10 242 (1.7)
 Post-traumatic stress disorder 144 758 (18.1) 121 388 (20.4)
 Major depressive disorder) 78 066 (9.8) 70 570 (11.9)
 Other depressive disorders 185 654 (23.2) 159 747 (26.9)
 General anxiety disorder 20 773 (2.6) 18 631 (3.1)
 Anxiety otherwise unspecified 81 315 (10.2) 67 058 (11.3)
 Other psychiatric diagnoses 12 912 (1.6) 8,548 (1.4)
Mean (SD) maximum daily morphine mg equivalents for opioid prescription 26.8 (35.0; 2 missing) 43.2 (66.1)
Opioid treatment type:
 No (%) long acting opioids 36 677 (4.6) 97 970 (16.5)
 No (%) short acting opioids 560 364 (70.1) 386 743 (65.0)
 No (%) tramadol only 202 627 (25.3) 109 721 (18.5)

SD=standard deviation.

In fiscal year 2013, 10% of patients were prescribed long acting opioids (n=134 647), 68% short acting opioids (n=947 107), and 22% tramadol only (n=312 348). Because we evaluated all patients with an outpatient prescription for an opioid analgesic (including those with short term prescriptions for acute postoperative or post-injury conditions), most were treated with short acting opioids, as expected. A higher proportion of patients who continued treatment with opioids were prescribed long acting opioids than those who stopped treatment with opioids. Maximum morphine milligram equivalents for an opioid prescription was nearly twice as high in patients who continued with their opioid treatment compared with those who stopped treatment.

We conducted a multivariable Cox non-proportional hazards regression model on 1 394 089 patients in fiscal year 2013 (13 patients were excluded because of missing data) with 2887 deaths from overdose or suicide (1851 overdose, 1249 suicide; deaths from overdose or suicide were not mutually exclusive). Thirty two per cent of patients were prescribed opioids for up to 30 days (n=445 918), 9% for 31-90 days (n=121 294), 23% for 91-400 days (n=316 520), and 36% for longer than 400 days (n=510 370). We found a significant interaction between stopping treatment with opioids and length of treatment that suggested a greater risk of death from overdose or suicide after stopping treatment the longer patients were treated with opioids (P<0.001; table 2). Specifically, the hazard ratios for patients who stopped opioid treatment (with reference values for all other covariates) were 1.67 (≤30 days), 2.80 (31-90 days), 3.95 (91-400 days), and 6.77 (>400 days). Other factors independently associated with a greater risk of death from overdose or suicide were prescriptions for long acting or short acting opioids compared with tramadol, log maximum daily morphine milligram equivalents, number of medical diagnoses, and having a diagnosis of a mental health or substance use disorder. Older age, female sex, and being married were independently associated with a lower risk of death from overdose or suicide.

Table 2.

Multivariable Cox non-proportional hazard regression model estimates for death from overdose or suicide in patients with an outpatient prescription for an opioid in fiscal year 2013 at the Veterans Health Administration

Variables Parameter estimate Standard error Hazard ratio (95% CI)
Stopped treatment with opioids 1.91 0.06 —*
Stopped treatment with opioids×duration of last prescription for opioids (reference >400 days):
 ≤30 days −1.40 0.08 —*
 31-90 days −0.88 0.11 —*
 91-400 days −0.54 0.08 —*
Age (centered) −0.02 0.002 0.98 (0.98 to 0.98)
Female sex −0.57 0.08 0.56 (0.48 to 0.66)
Currently married −0.63 0.04 0.53 (0.49 to 0.58)
Rural residence −0.07 0.04 0.93 (0.86 to 1.01)
No of medical diagnoses 0.02 0.01 1.02 (1.01 to 1.04)
Substance use disorder (excluding nicotine) 0.91 0.05 2.48 (2.25 to 2.72)
Nicotine use disorder 0.03 0.04 1.03 (0.95 to 1.12)
Mental health disorder 0.43 0.04 1.54 (1.41 to 1.68)
Type of opioid treatment (reference tramadol only):
 Long acting 0.44 0.08 1.55 (1.33 to 1.82)
 Short acting 0.15 0.06 1.16 (1.04 to 1.30)
Log maximum daily morphine mg equivalents 0.35 0.03 1.42 (1.34 to 1.50)
*

Hazard ratios and associated 95% confidence intervals were not estimated for these variables as this was an interaction model. To better understand the interaction, the estimated hazard ratios for patients who stopped opioid treatment (with reference values for all other covariates) were 1.67 (≤30 days), 2.80 (31-90 days), 3.95 (91-400 days), and 6.77 (>400 days).

Sensitivity analyses examining multivariable Cox non-proportional hazards regression models for death from overdose and suicide separately found the same general pattern of results as the combined model (appendix 3). Stopping treatment with opioids was associated with a greater risk of death from overdose and suicide the longer patients were treated with opioids. Specifically, for death from overdose, the hazard ratios for patients who stopped treatment with opioids (with reference values for all other covariates) were 1.39 (≤30 days), 2.41 (31-90 days), 3.25 (91-400 days), and 5.65 (>400 days), and the corresponding hazard ratios for death from suicide were 2.02 (≤30 days), 3.43 (31-90 days), 4.78 (91-400 days), and 7.99 (>400 days). In the combined model, we found no association between rural residence and death from overdose or suicide, whereas rural residence was associated with a lower risk of death from overdose but a higher risk of death from suicide in the separate models. The pattern of significant results was similar in the combined and overdose models, respectively, for age (P<0.001; P<0.001), number of medical diagnoses (P<0.05; P<0.001), and short acting opioids (P<0.01; P<0.05), but in the suicide model, age and short acting opioids were not significant. The number of medical diagnoses was associated with a lower risk in the suicide model (P<0.05) compared with a higher risk in the combined and overdose models.

Figure 1 (top panel) shows life table descriptive data over 375 days, with the conditional probabilities of death from overdose or suicide for patients treated with opioids who started an opioid prescription in fiscal year 2013 (n=952 918) and for patients who stopped treatment (n=799 668). The risk of death from overdose or suicide was increased for three months after starting or stopping opioid treatment. The incidence of death was increased more, however, in the three months after stopping treatment with opioids with the rates not stabilizing for almost six to 12 months after stopping. The full life table descriptive data for figure 1 (top panel) are in appendix 4.

Fig 1.

Fig 1

Conditional probability of death from overdose or suicide over 375 days (in 25 day intervals). Top panel=probability in patients treated with opioids (for patients who started treatment with opioids (n=952 918) in fiscal year 2013) and after stopping opioid treatment (n=799 668). Bottom panel=probability by length of treatment with opioids in those who stopped opioid treatment

Figure 1 (bottom panel) shows the life table descriptive data over 375 days, illustrating the interaction between stopping treatment with opioids and length of treatment. In the first 25 days after stopping treatment, deaths per 100 000 patients based on length of opioid treatment were 20 (≤30 days), 40 (31-90 days), 58 (91-400 days), and 104 (>400 days). The pattern of increased numbers of deaths in the first three months after stopping treatment with opioids for all time periods was similar to figure 1 (top panel), but stabilization across the longer times (31-90, 91-400, and >400 days) was not as clear, with variability observed from three to 12 months after stopping treatment with opioids. The full life table descriptive data for figure 1 (bottom panel) are in appendix 5.

Discussion

Principal findings

In this quality improvement evaluation, we examined the risk of death from overdose or suicide associated with stopping treatment with opioids and length of treatment, in patients in the Veterans Health Administration prescribed opioids in fiscal year 2013. We found that the interaction between stopping treatment and length of treatment with opioids was associated with an increased risk of death from overdose or suicide the longer patients had been treated with opioids before stopping. The risk was pronounced in patients treated the longest. Also independently associated with a greater risk of death from overdose or suicide were not being married, male sex, having a diagnosis of a mental health or substance use disorder, medical comorbidities, and higher doses of opioids, similar to previous reports.26 27 28 37 Our descriptive analyses showed the rise in the number of deaths from overdose or suicide in the months after stopping or starting opioid treatment, highlighting these vulnerable risk periods for patients. Our findings suggest that transitions (starting and stopping treatment) were associated with an increase in the risk for overdose and suicide for patients treated with opioids for any length of time, with increasing risk the longer patients were treated.

Comparisons with other studies

Previous studies focused mostly on smaller regional samples of patients treated with high doses of opioids or for long periods of time,17 18 whereas our study used national data from the Veterans Health Administration and included all patients prescribed opioids. All patients exposed to opioids had an increased risk of death from overdose or suicide after starting or stopping treatment with opioids. Although patients treated with opioids for long periods (eg, >400 days in our evaluation) had the highest hazard ratios after stopping treatment, even those treated for up to 30 days had a rise in the risk of death after treatment was stopped (hazard ratio of 1.4 for death from overdose after stopping opioid treatment and 1.7 for death from overdose or suicide). Those treated with opioids for 31-90 days had a hazard ratio of 2.4 for death from overdose after stopping opioid treatment (2.8 for death from overdose or suicide). Our study reinforces the importance of the first three months after stopping opioid treatment, extending the high risk period from 77 days17 to at least 100 days.

Priorities for future research

Future research should examine factors that might affect the risk or explain these findings. Glanz et al16 found that high dose variability was a major predictor of increased risk for opioid overdose, and the rate of tapering has been hypothesized to moderate the risk after stopping treatment. This is concerning given that Mark et al17 found that over half of patients treated with high doses of opioids for long periods of time suddenly stopped treatment with no reductions in dose (in one day), with 86% stopping within 21 days. The reasons for stopping treatment are also important. Demidenko et al15 found evidence of suicidal thoughts and self-harm in patients with and without substance use disorders whose clinicians stopped their long term treatment with opioids. James et al18 observed an increased risk of death from overdose in patients whose provider initiated stopping treatment although the findings were not statistically significant. In 77% of patients, at least one reason for stopping treatment came from the patients’ providers whereas in only 18% had the patient decided to stop treatment. The risk of stopping treatment might reflect worsening pain, conflicts with or disengagement from providers, particularly during exacerbations of mental health disorders (eg, aberrant behaviors, substance misuse, or mania), symptoms related to withdrawal, or a combination of these factors.15 38 39 Our findings suggest that future studies should include all patients prescribed opioids, not just those on long term treatment, and look at suicide behaviors in addition to outcomes of overdose.

Our evaluation did not focus solely on long term treatment with opioids or whether stopping treatment was voluntary, but our findings raise questions on whether evaluating stopping treatment with opioids in isolation is an effective population based strategy to reduce death from overdose or suicide. As we observed risks associated with starting and stopping treatment, our findings are consistent with: the recommendations in the Veterans Affairs/Department of Defense Clinical Practice Guideline against starting long term opioid treatment,13 the recently released guide for clinicians from the US Department of Health and Human Services on the appropriate dose reduction or discontinuation of long term opioid analgesics (HHS guide),40 41 an article by some of the lead authors of the guidelines from the Center for Disease Control and Prevention on long term prescribing of opioids42 highlighting the need for close monitoring and mitigation of risk after reducing or stopping treatment with opioids to avoid adverse outcomes, and the guidelines from the Veterans Affairs/Department of Defense and Center for Disease Control and Prevention to reduce reliance on opioids for the treatment of acute pain conditions.43 More research is needed to understand outcomes after stopping treatment with opioids in practice, particularly when to stop and which patients might benefit or be harmed by stopping treatment. Death from overdose or suicide was not limited to those with a prescription for opioids in this evaluation; patients who stopped treatment might have had unmanaged pain, or symptoms of opioid withdrawal and mood instability, which could have driven other risky strategies to obtain relief (eg, illicit opioids, sedating medications, self-harm). A better understanding of the risk after stopping treatment with opioids might help inform and improve the clinical care and outcomes for these patients.

Implications for clinical care

For clinical care, our findings suggest that patient centered efforts to mitigate the risk of death from overdose or suicide in patients prescribed opioids need to look at a broad array of risk factors for patients rather than focus solely on long term prescription management. Our findings suggest close monitoring for at least three months for any patient stopping treatment with opioids but especially in those on long term treatment or those with other risk factors (eg, mental health and substance use disorders). Pain conditions that required prescription of opioids, along with comorbid conditions, should be continually managed and treated. Surveillance, engagement with patients, and efforts to mitigate the risks are important when treatment is started and stopped, and especially in patients who have been treated with opioids for a long period of time before stopping.

Preservation of relationships with healthcare providers is important because the reasons for their concerns about prescribing opioids (eg, mental health or substance use disorders, polypharmacy, medical comorbidities, or social risks) might be the same issues that place patients at risk for overdose or suicide after stopping or starting treatment with opioids. Thus clinicians should avoid dismissing patients from care, ensure that patients continue to receive coordinated care, and have policies that allow them to account for each patient’s unique circumstances when making clinical decisions about prescribing opioids.40 41 42 Also, more detailed guidance and standard mitigation of risk after stopping or starting treatment with opioids (eg, guidance on prescription drug monitoring programs, urine drug screening, prescription of naloxone, frequency of follow-up appointments, evaluation of suicide risk) might be warranted. In general, implementing recommended safety strategies for starting patients on long term opioid treatment (eg, close monitoring, engagement with patients, pain management) might be relevant even for short term treatment, and is especially important after stopping treatment, particularly in those patients ending longer courses of opioids.

Limitations of the study

There are limitations to this evaluation. First, an observational evaluation cannot prove cause and effect and is subject to confounding. We cannot say from these data whether the increased risk was caused by a change in prescription; circumstances that triggered the decision to start or stop prescribing an opioid (eg, illness, injury, development of addiction, or other adverse events) might drive the increased risk of overdose or suicide. Second, our primary variable (stopping treatment with opioids) did not take into consideration the reasons or clinical intentions for stopping, or the speed of its execution.41 44 We did not record whether patients stopped treatment voluntarily or because their pain resolved, or if they were concerned about adverse outcomes. Moreover, because we included all patients prescribed opioid analgesics (a large proportion of whom were treated with short acting opioids), we expected our findings would be biased towards a lower risk associated with stopping treatment, because many patients stop taking opioids after recovery from short term injuries whereas long term pain is a risk factor for suicide.45 46 47 The interaction between stopping treatment with opioids and length of treatment suggested a lower risk for patients treated for shorter periods, but the risks were relatively high even in patients prescribed opioids for 90 days or less. Third, we lacked data on adherence to treatment, access to unused drugs after the estimated prescription period, and opioids obtained outside of the Veterans Health Administration. Fourth, to ensure that the risks were not related to restarting treatment with opioids after stopping, we examined the number of patients who had another opioid prescription in fiscal year 2014 after the date of stopping treatment in fiscal year 2013 to fiscal year 2014 (that is, after the date of stopping treatment for their last opioid prescription episode identified in fiscal year 2013 but within our observation period of fiscal year 2013 to fiscal year 2014). Of the 799 668 patients who stopped treatment with opioids in our evaluation, 4071 (0.5%) had another prescription for an opioid in fiscal year 2014 after their date of stopping treatment. None of these 4071 patients died of suicide or overdose during our evaluation period. Finally, our findings might not apply to other populations.

Conclusions

The patients in our evaluation were at greater risk of death from overdose or suicide after stopping treatment with opioids, with the risk increasing the longer patients had been treated before stopping. Our descriptive data suggested starting treatment with opioids is a risk period. In general, our evaluation identified the first three months after starting or stopping treatment with opioids as risk periods that offer clear opportunities for clinical intervention to improve the safety of prescribing opioids. These periods have not been a focus of guidelines for long term treatment with opioids.

Close monitoring of patients for up to one year after stopping treatment with opioids might be warranted. We suggest that efforts to reduce deaths from overdose and suicide take a broader view on patient safety, beyond opioid management, to mitigate the risk from the patient’s perspective. Efforts should look at risk factors that might place patients at risk for overdose or suicide whether they continue or stop treatment with opioid analgesics. Greater preventative measures and engagement with healthcare providers are needed in the first three months after starting or stopping treatment with opioids.

What is already known on this topic

  • In the US, a rise in deaths from overdose and opioid use disorders has prompted initiatives to reduce the risks associated with prescribing opioids, encouraging stopping treatment when risks exceed benefits

  • Studies have identified a rise in adverse events (death from overdose, suicidal thoughts, self-harm, and hospital admissions or emergency department visits for substance related events) after stopping treatment with opioids in patients treated long term

What this study adds

  • In patients prescribed opioids in the Veterans Health Administration, stopping treatment with opioids at any time had an increased risk of death from overdose or suicide, with the risk increasing the longer patients were treated

  • Efforts to mitigate the risk (patient monitoring, prevention of suicide and overdose) should be intensified for at least three months after starting or stopping treatment with opioids

Acknowledgments

We thank Eleanor Lewis for feedback on drafts of this paper and acknowledge the sacrifices of men and women who have served in the US military.

Web extra.

Extra material supplied by authors

Web appendix: Supplementary materials

olie051219.ww.pdf (730.1KB, pdf)

Contributors: EMO supervised the evaluation and is the guarantor. All authors contributed to the design and conduct of the evaluation, interpretation of the data, and preparation, review, and approval of the manuscript. TB, PH, AR, and MP assisted with data collection and management. TB and PH assisted with data analyses. EMO is the corresponding author and attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.

Funding: This project was supported by the Veterans Affairs Office of Mental Health and Suicide Prevention but the contents of this work do not represent the views of the Veterans Affairs or the US government.

Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf and declare: support from the Veterans Affairs Office of Mental Health Suicide Prevention for the submitted work; JMH received support from grants from the Office of the Director, National Institutes of Health (grant R01DA045027); SK owns stock in CVS Health, Thermo Fisher Scientific, and Zimmer Biomet, not exceeding 5% of his assets; he also personally owned stock in two pharmaceutical companies (Abbott and Merck) before 31 December 2017, which amounted to less than 5% of his assets, and were sold before the end of 2017. SK’s spouse personally owns stock in Abbott, Merck, and Johnson & Johnson, not exceeding 10% of her assets. SK has offered opinions in the peer reviewed and lay literature on the ethics of non-consensual or suddenly stopping opioids in patients with pain. All other authors declare no financial relationships with any organizations that might have an interest in the submitted work in the previous three years, and no other relationships or activities that could appear to have influenced the submitted work.

Ethical approval: Not required. This quality improvement project was not considered research by the institutional review board of Stanford University.

Data sharing: No additional data available.

The lead author (the manuscript’s guarantor) affirms that the manuscript is an honest, accurate, and transparent account of the evaluation being reported; that no important aspects of the evaluation have been omitted; and that any discrepancies from the evaluation as originally planned (and, if relevant, registered) have been explained.

Dissemination to participants and related patient and public communities: We shared preliminary findings from an early version of this evaluation in the Fall 2018 with the US Food and Drug Administration when the agency was looking into the risk associated with stopping treatment with opioids (note: the US Food and Drug Administration released a safety announcement on 9 April 2019, on the harms of sudden discontinuation of opioids (www.fda.gov/drugs/drug-safety-and-availability/fda-identifies-harm-reported-sudden-discontinuation-opioid-pain-medicines-and-requires-label-changes) which received substantial public attention). Findings and recommendations from this study will be incorporated into the Veterans Health Administration’s nationally standardized patient and provider educational materials and computerized decision support systems designed to support risk-benefit and opioid management discussions for patients on or considering opioid prescriptions. These standardized tools are used in required informed consent discussions for chronic opioid treatment and risk reviews before starting opioids and are provided to support ongoing management of patients treated with opioids. At the time of acceptance of the manuscript, this revision process is in progress and has been partially implemented. The authors will work with the Veterans Health Administration Communications to develop a press release on the findings to increase public awareness, subsequent to publication of the manuscript.

References

  • 1. Rudd RA, Seth P, David F, Scholl L. Increases in drug and opioid-involved overdose deaths - United States, 2010-2015. MMWR Morb Mortal Wkly Rep 2016;65:1445-52. 10.15585/mmwr.mm655051e1  [DOI] [PubMed] [Google Scholar]
  • 2. Saha TD, Kerridge BT, Goldstein RB, et al. Nonmedical prescription opioid use and DSM-5 nonmedical prescription opioid use disorder in the United States. J Clin Psychiatry 2016;77:772-80. 10.4088/JCP.15m10386  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3. Mokdad AH, Ballestros K, Echko M, et al. US Burden of Disease Collaborators The state of US health, 1990-2016: burden of diseases, injuries, and risk factors among US states. JAMA 2018;319:1444-72. 10.1001/jama.2018.0158  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4. Martins SS, Sarvet A, Santaella-Tenorio J, Saha T, Grant BF, Hasin DS. Changes in US lifetime heroin use and heroin use disorder: prevalence from the 2001-2002 to 2012-2013 National Epidemiologic Survey on Alcohol and Related Conditions. JAMA Psychiatry 2017;74:445-55. 10.1001/jamapsychiatry.2017.0113  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.St Amour M. Central Maine patients fear weaning off opioids as they struggle with chronic pain. 21 January 2017. https://www.centralmaine.com/2017/01/21/central-maine-patients-fear-medication-weaning-as-they-struggle-with-chronic-pain/
  • 6.Davis C. State by state summary of opioid prescribing regulations and guidelines. 2017. https://www.azdhs.gov/documents/prevention/womens-childrens-health/injury-prevention/opioid-prevention/appendix-b-state-by-state-summary.pdf
  • 7. García MC, Dodek AB, Kowalski T, et al. Declines in opioid prescribing after a private insurer policy change - Massachusetts, 2011-2015. MMWR Morb Mortal Wkly Rep 2016;65:1125-31. 10.15585/mmwr.mm6541a1  [DOI] [PubMed] [Google Scholar]
  • 8.Centers for Medicare and Medicaid Services. Centers for Medicare and Medicaid Services (CMS) opioid misuse strategy 2016. United States Department of Health and Human Services; 5 January 2017. https://www.cms.gov/Outreach-and-Education/Outreach/Partnerships/Downloads/CMS-Opioid-Misuse-Strategy-2016.pdf
  • 9. Murthy VH. Ending the opioid epidemic - a call to action. N Engl J Med 2016;375:2413-5. 10.1056/NEJMp1612578  [DOI] [PubMed] [Google Scholar]
  • 10. Dowell D, Haegerich TM, Chou R. CDC guideline for prescribing opioids for chronic pain - United States, 2016. MMWR Recomm Rep 2016;65:1-49. 10.15585/mmwr.rr6501e1  [DOI] [PubMed] [Google Scholar]
  • 11. Meara E, Horwitz JR, Powell W, et al. State legal restrictions and prescription-opioid use among disabled adults. N Engl J Med 2016;375:44-53. 10.1056/NEJMsa1514387  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12. Kertesz SG. Turning the tide or riptide? The changing opioid epidemic. Subst Abus 2017;38:3-8. [DOI] [PubMed] [Google Scholar]
  • 13.Opioid Therapy for Chronic Pain Work Group. VA/DoD clinical practice guideline for opioid therapy for chronic pain. Washington, DC: Department of Veterans Affairs Department of Defense; 1 March 2017.
  • 14. Bohnert ASB, Guy GP, Jr, Losby JL. Opioid prescribing in the United States before and after the Centers for Disease Control and Prevention’s 2016 opioid guideline. Ann Intern Med 2018;169:367-75. 10.7326/M18-1243  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15. Demidenko MI, Dobscha SK, Morasco BJ, Meath THA, Ilgen MA, Lovejoy TI. Suicidal ideation and suicidal self-directed violence following clinician-initiated prescription opioid discontinuation among long-term opioid users. Gen Hosp Psychiatry 2017;47:29-35. 10.1016/j.genhosppsych.2017.04.011  [DOI] [PubMed] [Google Scholar]
  • 16. Glanz JM, Binswanger IA, Shetterly SM, Narwaney KJ, Xu S. Association between opioid dose variability and opioid overdose among adults prescribed long-term opioid therapy. JAMA Netw Open 2019;2:e192613. 10.1001/jamanetworkopen.2019.2613  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17. Mark TL, Parish W. Opioid medication discontinuation and risk of adverse opioid-related health care events. J Subst Abuse Treat 2019;103:58-63. 10.1016/j.jsat.2019.05.001  [DOI] [PubMed] [Google Scholar]
  • 18. James JR, Scott JM, Klein JW, et al. Mortality after discontinuation of primary care-based chronic opioid therapy for pain: a retrospective cohort study. J Gen Intern Med 2019;34:2749-55. 10.1007/s11606-019-05301-2  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19. Breckenridge J, Clark JD. Patient characteristics associated with opioid versus nonsteroidal anti-inflammatory drug management of chronic low back pain. J Pain 2003;4:344-50. 10.1016/S1526-5900(03)00638-2  [DOI] [PubMed] [Google Scholar]
  • 20. Sullivan MD, Edlund MJ, Zhang L, Unützer J, Wells KB. Association between mental health disorders, problem drug use, and regular prescription opioid use. Arch Intern Med 2006;166:2087-93. 10.1001/archinte.166.19.2087  [DOI] [PubMed] [Google Scholar]
  • 21. Oliva EM, Bowe T, Tavakoli S, et al. Development and applications of the Veterans Health Administration’s Stratification Tool for Opioid Risk Mitigation (STORM) to improve opioid safety and prevent overdose and suicide. Psychol Serv 2017;14:34-49. 10.1037/ser0000099  [DOI] [PubMed] [Google Scholar]
  • 22. Frank JW, Lovejoy TI, Becker WC, et al. Patient outcomes in dose reduction or discontinuation of long-term opioid therapy: a systematic review. Ann Intern Med 2017;167:181-91. 10.7326/M17-0598  [DOI] [PubMed] [Google Scholar]
  • 23. Gellad WF, Good CB, Shulkin DJ. Addressing the opioid epidemic in the United States: lessons from the Department of Veterans Affairs. JAMA Intern Med 2017;177:611-2. 10.1001/jamainternmed.2017.0147  [DOI] [PubMed] [Google Scholar]
  • 24. Lin LA, Bohnert ASB, Kerns RD, Clay MA, Ganoczy D, Ilgen MA. Impact of the opioid safety initiative on opioid-related prescribing in veterans. Pain 2017;158:833-9. 10.1097/j.pain.0000000000000837  [DOI] [PubMed] [Google Scholar]
  • 25. Hadlandsmyth K, Mosher H, Vander Weg MW, Lund BC. Decline in prescription opioids attributable to decreases in long-term use: a retrospective study in the Veterans Health Administration 2010-2016. J Gen Intern Med 2018;33:818-24. 10.1007/s11606-017-4283-8  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26. Bohnert ASB, Valenstein M, Bair MJ, et al. Association between opioid prescribing patterns and opioid overdose-related deaths. JAMA 2011;305:1315-21. 10.1001/jama.2011.370  [DOI] [PubMed] [Google Scholar]
  • 27. Fulton-Kehoe D, Sullivan MD, Turner JA, et al. Opioid poisonings in Washington State Medicaid: trends, dosing, and guidelines. Med Care 2015;53:679-85. 10.1097/MLR.0000000000000384  [DOI] [PubMed] [Google Scholar]
  • 28. Zedler B, Xie L, Wang L, et al. Risk factors for serious prescription opioid-related toxicity or overdose among Veterans Health Administration patients. Pain Med 2014;15:1911-29. 10.1111/pme.12480  [DOI] [PubMed] [Google Scholar]
  • 29. Elixhauser A, Steiner C, Harris DR, Coffey RM. Comorbidity measures for use with administrative data. Med Care 1998;36:8-27. 10.1097/00005650-199801000-00004  [DOI] [PubMed] [Google Scholar]
  • 30.Centers for Disease Control and Prevention. Calculating total daily dose of opioids for safer dosage. https://www.cdc.gov/drugoverdose/pdf/calculating_total_daily_dose-a.pdf. 2019.
  • 31. Oliva EM, Bowe T, Tavakoli S, et al. Development and applications of the Veterans Health Administration’s Stratification Tool for Opioid Risk Mitigation (STORM) to improve opioid safety and prevent overdose and suicide. Psychol Serv 2017;14:34-49. 10.1037/ser0000099  [DOI] [PubMed] [Google Scholar]
  • 32. Midboe AM, Lewis ET, Paik MC, et al. Measurement of adherence to clinical practice guidelines for opioid therapy for chronic pain. Transl Behav Med 2012;2:57-64. 10.1007/s13142-011-0104-5  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33. Borucka J. Extensions of Cox model for non-proportional hazards purpose. Ekonometria 2014;3:85-101 10.15611/ekt.2014.3.07. [DOI] [Google Scholar]
  • 34.Wilson M. Using SAS to assess and model time-to-event data with non-proportional hazards. Paper presented at: Proceedings of the Midwest SAS User Group Conference 2010. [Google Scholar]
  • 35. Crowley J, Hu M. Covariance analysis of heart transplant survival data. J Am Stat Assoc 1977;72:27-36 10.1080/01621459.1977.10479903 . [DOI] [Google Scholar]
  • 36. Suissa S. Immortal time bias in pharmaco-epidemiology. Am J Epidemiol 2008;167:492-9. 10.1093/aje/kwm324  [DOI] [PubMed] [Google Scholar]
  • 37. Gomes T, Mamdani MM, Dhalla IA, Paterson JM, Juurlink DN. Opioid dose and drug-related mortality in patients with nonmalignant pain. Arch Intern Med 2011;171:686-91. 10.1001/archinternmed.2011.117  [DOI] [PubMed] [Google Scholar]
  • 38. Lovejoy TI, Morasco BJ, Demidenko MI, Meath TH, Frank JW, Dobscha SK. Reasons for discontinuation of long-term opioid therapy in patients with and without substance use disorders. Pain 2017;158:526-34. 10.1097/j.pain.0000000000000796  [DOI] [PubMed] [Google Scholar]
  • 39. Jawad I, Watson S, Haddad PM, Talbot PS, McAllister-Williams RH. Medication nonadherence in bipolar disorder: a narrative review. Ther Adv Psychopharmacol 2018;8:349-63. 10.1177/2045125318804364  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.US Department of Health and Human Services. HHS guide for clinicians on the appropriate dosage reduction or discontinuation of long-term opioid analgesics. 2019. https://www.hhs.gov/opioids/treatment/clinicians-guide-opioid-dosage-reduction/index.html.
  • 41. Dowell D, Compton WM, Giroir BP. Patient-centered reduction or discontinuation of long-term opioid analgesics: the HHS guide for clinicians. JAMA 2019;1-3. 10.1001/jama.2019.16409  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42. Dowell D, Haegerich T, Chou R. No shortcuts to safer opioid prescribing. N Engl J Med 2019;380:2285-7. 10.1056/NEJMp1904190  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43. Bohnert ASB, Ilgen MA. Understanding links among opioid use, overdose, and suicide. N Engl J Med 2019;380:71-9. 10.1056/NEJMra1802148  [DOI] [PubMed] [Google Scholar]
  • 44. Mark TL, Parish W. Opioid medication discontinuation and risk of adverse opioid-related health care events. J Subst Abuse Treat 2019;103:58-63. 10.1016/j.jsat.2019.05.001  [DOI] [PubMed] [Google Scholar]
  • 45. de Heer EW, Ten Have M, van Marwijk HWJ, et al. Pain as a risk factor for suicidal ideation. A population-based longitudinal cohort study. Gen Hosp Psychiatry 2018;S0163-8343(18)30027-6. 10.1016/j.genhosppsych.2018.11.005  [DOI] [PubMed] [Google Scholar]
  • 46. Racine M. Chronic pain and suicide risk: A comprehensive review. Prog Neuropsychopharmacol Biol Psychiatry 2018;87(Pt B):269-80. 10.1016/j.pnpbp.2017.08.020  [DOI] [PubMed] [Google Scholar]
  • 47. Ilgen MA, Zivin K, McCammon RJ, Valenstein M. Pain and suicidal thoughts, plans and attempts in the United States. Gen Hosp Psychiatry 2008;30:521-7. 10.1016/j.genhosppsych.2008.09.003  [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Web appendix: Supplementary materials

olie051219.ww.pdf (730.1KB, pdf)

Articles from The BMJ are provided here courtesy of BMJ Publishing Group

RESOURCES