Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2016 Mar 1.
Published in final edited form as: JAMA Intern Med. 2015 Mar 1;175(3):420–427. doi: 10.1001/jamainternmed.2014.6294

Out-of-Hospital Mortality among Patients Receiving Methadone for Non-Cancer Pain

Wayne A Ray 1, Cecilia P Chung 1, Katherine T Murray 1, William O Cooper 1, Kathi Hall 1, C Michael Stein 1
PMCID: PMC4346542  NIHMSID: NIHMS641066  PMID: 25599329

Abstract

Importance

Growing methadone use in pain management has raised concerns regarding its safety relative to other long-acting opioids. Methadone may increase risk for both lethal respiratory depression related to accidental overdose and life-threatening ventricular arrhythmias.

Objective

To compare risk of out-of-hospital death in users of methadone for non-cancer pain to that for comparable users of sustained-release (SR) morphine.

Design

Retrospective cohort study.

Setting

Tennessee Medicaid, 1997 through 2009.

Participants

Cohort included current users of morphine SR or methadone 30–74 years of age without cancer or other life-threatening illness and not in a hospital or nursing home. At cohort entry, 32,742 and 6,014 had filled a prescription for morphine SR or methadone, respectively. The median age was 48 years, 58% were female, and comparable proportions had received cardiovascular, psychotropic, and other musculoskeletal medications. Nearly 90% of patients received the opioid for either back or other musculoskeletal pain. The median daily doses prescribed for morphine SR and methadone were 90mg and 40mg, respectively.

Main Outcomes and Measures

The primary study endpoint was out-of-hospital mortality, given that opioid-related deaths typically occur outside the hospital.

Results

There were 477 deaths during 28,699 person years of followup, or 166 deaths per 10,000 person-years. After control for study covariates, current methadone users had a 46% increased risk of death during followup, with an adjusted hazard ratio (HR) of 1.46 (95% confidence interval 1.17–1.83, p = .0008), resulting in 72 (27–130) excess deaths per 10,000 person-years. Methadone users of doses ≤20mg/day, the lowest dose quartile, had increased risk (HR =1.59 [1.01–2.51], p = .0461) relative to a comparable dose of morphine SR (<60mg/day).

Conclusions and Relevance

The increased risk of death observed for users of methadone, even for low doses, supports recommendations that it should not be a drug of first choice for non-cancer pain.


Methadone, a µ-opioid agonist long used as evidence-based treatment for opioid dependence,1 has been increasingly prescribed for chronic pain. In 2009, 4.4 million methadone prescriptions in the U.S. were for treatment of pain, accounting for 9% of prescribed opioid analgesics on a dose-adjusted basis.1 Methadone's primary advantages as an analgesic are a long elimination half-life,2 and low cost;1 however, its efficacy is comparable to that of other long-acting opioids.3

There are major concerns regarding methadone's relative safety. The risk for accidental overdose and lethal respiratory depression may be greater than that for other long-acting opioids. Because the duration of methadone's respiratory depressant effects is longer than that for its analgesic effects,4;5 inadvertent intoxication can occur as dose is increased to provide greater pain relief. This risk may be exacerbated by methadone's highly variable pharmacokinetics.4;5 In 2006, the FDA issued an advisory and the label was modified to warn of the potential for unintentional overdose.57 This concern was reinforced by autopsy series of opioid overdose deaths with over-representation of methadone-related cases8;9 and a U.S. study demonstrating a disproportionate number of prescription-opioid-related overdose deaths with methadone involvement.1

Methadone also has adverse cardiac effects. It prolongs the QT interval10 and has been implicated in numerous case reports of life-threatening ventricular arrhythmias.1013 Cases of sudden cardiac death, the majority of which are due to ventricular arrhythmias,14;15 have been reported in methadone patients.16

These data have led to questions regarding the appropriateness of the widespread use of methadone for the treatment of chronic pain, particularly given other equally effective alternatives.1;3;17 However, the one cohort study comparing methadone to sustained-release (SR) morphine unexpectedly found that adjusted overall mortality was 44% lower for the methadone users.18 Given this controversy, we conducted a cohort study of patients receiving either methadone or morphine SR for non-cancer pain. Given the multiple mechanisms by which an opioid could increase mortality, the primary endpoint was total mortality during study followup.

Methods

Cohort and Followup

We conducted a retrospective cohort study of Tennessee Medicaid enrollees with a filled prescription for methadone or morphine SR from 1997 through 2009. The Medicaid files provided an efficient source of data for identifying the cohort, determining periods of probable exposure to medications, and ascertaining deaths.19;20 The study Medicaid database included enrollment, pharmacy, hospital, outpatient, and nursing home files and was augmented with linkage to death certificates19;21 and a statewide hospital discharge database.

To improve the study capacity to identify deaths related to opioids and thus to reduce the potential for confounding, we focused on deaths outside the hospital in patients for whom such deaths should otherwise be relatively infrequent. Thus, to decrease the likelihood of deaths related to terminal illness, the cohort excluded persons 75 years of age or older, patients with cancer and other life-threatening diseases, and nursing home residents (Appendix Table 1). Patients in the hospital could not enter the cohort until 30 days after discharge, because deaths during this period may be related to the reasons for the hospitalization. We excluded persons with recorded evidence of drug abuse, given the increased risk for opioid overdose unrelated to therapeutic use.

Patients entered the cohort on the date of filling of the first prescription for methadone or morphine SR on which they met the study inclusion/exclusion criteria (Appendix Table 1). They remained in the cohort until the end of the study, death, failure to meet inclusion/exclusion criteria, or the cessation of study opioid use. Patients who left the cohort could reenter if they subsequently became eligible.

Given that both respiratory depression and cardiac arrhythmias are acute drug effects,22;23 study followup consisted of current use of study opioids (Appendix Figure 1). Each study opioid prescription contributed a period of current drug use (length equal to prescription days of supply) to study followup. Persons prescribed both study opioids during followup contributed person-time to both the morphine SR and methadone categories, although overlapping use was not permitted (Appendix Figure 1).

We excluded person-time during and in the 30 days following hospitalization to decrease the likelihood of deaths unrelated to opioid toxicity and thus both reduce potential confounding and improve our capacity to detect an adverse effect of opioids. This could introduce bias if the study groups differed with regard to the proportions of patients with a lethal opioid adverse effect who survived until hospital admission but ultimately died in the hospital. However, this scenario seems unlikely, given that opioid-related deaths typically occur outside the hospital. Opioid-related respiratory depression is infrequently fatal in patients administered an opioid antagonist.24 Torsade de pointes is rapidly lethal, most frequently leading to death before the patient can seek medical care.14;15;25

Endpoints

The primary endpoint was all deaths during study followup. To provide insight into the potential mechanisms for opioid toxicity, deaths were classified into three subgroups: 1) sudden unexpected deaths consistent with either opioid overdose or life-threatening arrhythmias; 2) other respiratory/cardiovascular deaths for which opioid involvement was possible but less certain, and 3) other deaths which were less likely to be related to opioid toxicity. Classification was based on the death certificate underlying cause of death, adjudication of terminal medical records, and computerized files with both terminal medical encounters and death certificate information (Appendix).

Sudden unexpected deaths met previous definitions for either opioid overdose1;24 or sudden cardiac death (Appendix).10;23;26;27 These were consistent with either opioid-related respiratory depression or cardiac arrhythmias, typically occurred unexpectedly within a short interval in persons in a usual state of health and had no evident cause other than opioid toxicity. Opioid overdose deaths were identified from the death certificate underlying cause of death codes (Appendix Table 2). Prior studies suggest these codes have a positive predictive value greater than 90%.28 Sudden cardiac deaths10 were identified from adjudication of terminal medical records,23;26;27 or, when these were unavailable, from a previously validated computer-based definition with a positive predictive value of 87%-90% (Appendix).23;26

Given the FDA focus on unintentional methadone overdose,57 we sought to identify opioid overdose deaths. However, the clinical circumstances of overdose and sudden cardiac death often are similar (e.g. unexpected death during sleep) and it can be difficult to distinguish these mechanisms post mortem.11;13 Thus, when terminal medical records were available for adjudication, a death with a coded underlying cause of death of opioid overdose also could meet the definition for sudden cardiac death (Appendix). Given this, sudden unexpected deaths were further classified as: meeting the definition for opioid overdose only, meeting the definition for sudden cardiac death only, and meeting both definitions. We also performed sensitivity analyses in which these definitions were mutually exclusive.

Other respiratory/cardiovascular deaths did not meet the definition for sudden unexpected death but potentially were related to adverse respiratory or cardiac drug effects (Appendix). These included deaths coded as due to respiratory causes, for which opioids may play a role.29 They also included deaths coded as due to cardiovascular causes for which sudden cardiac death had not been ruled out because medical records were unavailable (Appendix).

Other deaths during followup were considered as less likely to be related to opioid toxicity. These were predominantly injury deaths, cardiovascular deaths for which sudden cardiac death was deemed unlikely following medical record adjudication, alcohol-related deaths, and deaths due to non-respiratory infections.

Statistical Analysis

The relative risk of death between groups defined by study opioid use status, adjusted for patient characteristics, was estimated with the hazard ratio (HR) from a proportional hazards regression model, with study opioid use as a time-dependent covariate. A single person could have both morphine SR and methadone current use person-time in the analysis (Appendix Figure 1). However, for each person, these time periods were non-overlapping and the endpoint (death) occurred only once. Thus, statistical independence assumptions were not violated.30

The HRs were adjusted for potential differences between current users of methadone and morphine SR. Patient characteristics were described by 196 covariates (Appendix Table 3), which included calendar time, demographic factors, opioid indication (as previously defined31, see Appendix), use and dose of nonstudy opioids, cardiovascular medications and diagnoses, psychiatric medications and diagnoses, medications for musculoskeletal disorders, respiratory conditions, indicators of frailty, other pro-arrythmic medications, other comorbidity, and recent medical care utilization. Given that patient comorbidity could vary markedly during followup, covariates were updated at the time of each prescription fill.

Given the large number of study covariates, we controlled for these by stratifying the regression analyses (Appendix) by deciles of either a propensity score3234 or a mortality risk score,3436 both of which were time-dependent. The propensity score, used for comparisons of methadone vs morphine SR with no exposure subcategories, was the estimated probability that a study opioid prescription was for methadone given the covariate values at the time of the prescription fill. The distributions of the propensity score for methadone and morphine overlapped substantially (Appendix Table 4).

The mortality risk score (Appendix Figure 3) was a disease risk score3436 for total study mortality. It was the expected risk of death during followup as a function of the study covariates, in the absence of methadone use. Disease risk scores facilitate multiple exposure category analyses (such as dose-specific comparisons), given that propensity scores are less suited to non-binary comparisons.3436

We analyzed the risk of death according to quartiles of dose of study opioid therapy at the time of each prescription fill. We identified equivalent doses according to the cohort dose percentile distribution, which guarded against possible non-linear dose-equivalence of methadone and morphine SR. Tests for dose-response or tests of a methadone effect within categories defined by dose were performed with appropriate contrasts (Appendix).

We conducted several sensitivity analyses. We assessed the effect of non-proportional hazards by including a time by study opioid interaction in the model (Allison,37 pp. 177–179). We performed an analysis in which each methadone user was propensity-score-matched to a morphine SR user in which the absolute difference in key covariate prevalence between the two groups was never greater than 1% (Appendix Table 5). We analyzed several important subgroups, including those defined by calendar year, restricting followup to the first year of study opioid use, censoring of person-time on switching of study opioids, having age less than 65 years, having a known opioid indication, and being a new user38 of study opioids. The latter group was defined as patients with no prescription for a study opioid filled in the 31 to 365 days preceding cohort entry. We permitted prescriptions in the 30 days preceding cohort entry because this commonly occurred for patients initiating long-acting opioid therapy following hospitalization, who could not enter the study cohort until 30 days following discharge.

We estimated the absolute difference in the risk of death between methadone and morphine SR. The annual incidence of total study mortality for morphine SR was unadjusted; that for methadone was calculated by multiplying the incidence in the morphine group by the HR for the entire study population.

All analyses were done with SAS version 9.3. All p-values are two-sided. The Vanderbilt Institutional Review Board approved the study. The study was funded by federal agencies with no role in study conduct or reporting. The listed authors were entirely responsible for study design, data analysis, manuscript preparation, and publication decisions. The first manuscript draft was written by the primary author, who vouches for the data and the analysis.

Results

The cohort included 38,756 persons. At the time of cohort entry, 32,742 and 6,014 had filled a prescription for morphine SR or methadone, respectively, of which 75.7% and 71.9% were new users. The median age was 48 years, 58% were female, and similar proportions had received cardiovascular, psychotropic, and other musculoskeletal medications (Table 1). The indication for the study opioid was either back or other musculoskeletal pain for nearly 90% of patients. The median daily doses prescribed for morphine SR and methadone were 90mg and 40mg, respectively. There was concurrent use of non-study opioids for 65% and 48% of morphine SR and methadone patients, respectively. Comparable proportions of morphine SR and methadone patients had received cardiovascular, psychotropic, and other musculoskeletal medications.

Table 1.

Cohort characteristics. Unless otherwise noted, all values are proportions. Characteristics as of the time of filling of prescriptions included in cohort followup. Unless otherwise stated, medication variables reflect use on the study opioid prescription fill date and during the preceding 365 days.

Morphine SR Methadone
Cohort members, Na 32,742 6,014
Study opioid prescriptions during followup, N 355,687 76,035
Year of prescription, , median (inter-quartile range) 2004 (2002–2006) 2004 (2002–2005)
Age, years, median (inter-quartile range) 48 (42–55) 47 (41–54)
Female 57.9% 57.8%
White 84.4% 84.5%
Medicaid enrollment disabled 69.6% 68.3%
Study opioid prescription characteristics
Opioid indication
   Back pain 78.2% 77.2%
   Other musculoskeletal pain 11.8% 10.7%
   Other chronic pain: soft tissue, abdominal, or neurologic 2.8% 3.9%
   Acute pain related to medical/dental procedure, trauma, or infection 1.5% 1.4%
   No indication recorded 5.7% 6.8%
Dose per day, mg
   Median (inter-quartile range) 90 (60–160) 40 (20–60)
   Greater than 100mg methadone or 200mg morphine SR 12.0% 11.4%
New user of study opioids 75.7% 71.9%
Non-study opioids, current use 65.4% 47.7%
Psychotropic medications
Benzodiazepine 63.4% 64.2%
Cyclic antidepressant 26.4% 25.6%
SSRI or other antidepressant 63.6% 67.2%
Antipsychotic, % 16.0% 18.1%
Cardiovascular medications
Digoxin 2.2% 1.6%
Loop diuretic 21.7% 22.4%
Insulin or oral hypoglycemic 18.6% 18.1%
Musculoskeletal medications
NSAID/coxib 62.7% 63.0%
Musculoskeletal relaxant 65.1% 64.9%
Corticosteroid, systemic 5.9% 5.1%
Respiratory medications and diagnoses
Beta agonist 33.9% 30.7%
Other bronchodilator 20.1% 17.9%
COPD 23.8% 18.5%
Asthma 11.2% 10.8%
Home oxygen 8.9% 6.5%
Medical care utilization
Hospital stay past year 18.2% 16.9%
ED visit past 30 days 9.5% 8.5%
Overdose ED/inpatient past year 3.4% 3.6%
a

As of the first day of study followup. During followup, there were 2473 initial morphine SR users who also received methadone during followup and 1453 initial methadone users who subsequently received morphine SR.

There were 477 deaths during 28,699 person years of cohort followup, or 166 deaths per 10,000 person-years. Of the study deaths, 346 (72.5%) were sudden unexpected deaths, 53 (11.1%) were other respiratory/cardiovascular deaths, and 78 (16.4%) were other deaths.

After adjustment for study covariates, current methadone users had a 46% increased risk of death during followup (Figure 1 and Table 2, HR [95% CI] = 1.46 [1.17–1.83], p = .0008). Methadone users had 72 (27–130) excess deaths per 10,000 person-years of followup.

Figure 1.

Figure 1

Adjusted cumulative incidence of deaths during followup according to study opioid use. Data shown for first 4 years of followup, the period during which all deaths in the methadone group occurred. HR denotes the adjusted hazard ratio (calculated for the entire period of study followup), the 95% confidence interval is in parentheses. The cumulative incidence for morphine SR, the reference group, is unadjusted; that for methadone is adjusted by multiplying the unadjusted incidence by HRA/HRu, the ratio of the adjusted:unadjusted hazard ratios. SR denotes sustained release.

Table 2.

Risk of death during current use of morphine sustained release (SR) and methadone, by type of death. HR denotes adjusted hazard ratio; CI denotes confidence interval.

Morphine SR Methadone
Study followup

Person-years 23,609 5,091

Deaths

Rate/10,000 person-years (N deaths) HR (95% CI) p-value
All deaths 156.3 (369) 212.2 (108) 1.46 (1.17–1.83) 0.0008
Sudden unexpected death 112.2 (265) 159.1 (81) 1.47 (1.13–1.90) 0.0037
   Meeting definition for opioid overdose only 11.4 (27) 31.4 (16) 2.54 (1.33–4.84) 0.0046
   Meeting definition for sudden cardiac death only 78.8 (186) 82.5 (42) 1.12 (0.80–1.59) 0.5067
   Meeting definition for both opioid overdose and sudden cardiac death 22.0 (52) 45.2 (23) 2.02 (1.21–3.37) 0.0070
Other respiratory/cardiovascular deaths 17.4 (41) 23.6 (12) 1.78 (0.91–3.46) 0.0897
Other deathsa 26.7 (63) 29.5 (15) 1.26 (0.70 –2.26) 0.4474
a

Includes 31 injury deaths, 14 cardiovascular deaths for which sudden cardiac death was ruled out on adjudication of medical records, 5 deaths related to alcohol, 4 infection deaths, and 24 deaths from a range of other causes.

Current methadone users had increased risk for sudden unexpected death (Table 2, HR = 1.47 [1.13–1.90], p = .0037). For deaths meeting the definition for opioid overdose, methadone users had more than a two-fold increased risk (Table 2). There was no significantly increased risk for deaths meeting the definition for sudden cardiac death but not for opioid overdose (HR = 1.12 [0.80–1.59], p =.5067). A sensitivity analysis with mutually exclusive definitions for opioid overdose and sudden cardiac death had essentially similar findings, as did an analysis restricted to adjudicated deaths (Appendix Figure 4 and Table 6).

Current methadone users also had increased risk for other respiratory/cardiovascular deaths (HR = 1.78 [0.91–3.46]), but this was of borderline statistical significance (p = .0897). The HR for other deaths was 1.26 (0.70–2.26, p = .4474).

The risk of death during followup increased with increasing dose for both study opioids (Figure 2, p = .0165); the dose-response trends for the two study opioids did not differ significantly (p=.3258). Current users of methadone with doses below the median had significantly greater risk of death than did comparable current users of morphine SR (HR = 1.70 [1.23–2.34], p = .0014); for doses above the median risk was increased for methadone users but the significance was borderline (HR = 1.31 [0.98–1.74], p = .0709). Methadone users in the lowest dose quartile, ≤20mg/day, had greater risk of death during followup than did users of comparable doses of morphine SR (HR =1.59 [1.01–2.51], p = .0461).

Figure 2.

Figure 2

Adjusted annual incidence of study deaths according to study opioid use and methadone daily dose quartile. Bars are 95% confidence interval for incidence rates. HR denotes the adjusted methadone vs morphine hazard ratio. The 95% confidence interval is in parentheses.

We assessed the robustness of the primary study finding in an analysis that controlled for non-proportional hazards, in a propensity-score-matched analysis, and in analyses restricted to important subgroups, including new users of the study opioids (Table 3). In each of these analyses the HR for methadone current use was increased and statistically significant. There was no evidence that the magnitude of the HRs differed from those of the primary analysis: in every case the HR for the primary analysis was included in the confidence interval for the HR from the sensitivity analysis.

Table 3.

Sensitivity analyses.

Methadone vs Morphine
SR: All Deaths HR
95% CI p-value
Entire cohort, primary analysis 1.46 1.17–1.83 0.0008
Entire cohort, controls for non-proportional hazards 1.75 1.33–2.31 <0.0001
Propensity-score matched morphine SR users 1.61 1.19–2.18 0.0021
Calendar year
   1997–2003 1.40 1.01–1.94 0.0438
   2004–2009 1.51 1.11–2.04 0.0083
Followup restricted to first year of study opioid use 1.81 1.39–2.36 <0.0001
Censoring on switching to different study opioid 1.32 1.02–1.70 0.0372
Age <65 years 1.54 1.23–1.94 0.0002
Restricted to known study opioid indication 1.57 1.25–1.98 0.0001
New users of study opioid 1.38 1.06–1.80 0.0168

Comment

We identified a Tennessee Medicaid cohort of morphine SR and methadone users receiving treatment for non-cancer pain using inclusion/exclusion criteria designed to minimize the risk of deaths unrelated to opioid adverse effects. The methadone users had a 46% increased risk for mortality during study followup, or an absolute excess risk of 72 deaths per 10,000 years of followup. The increased risk for methadone was present for doses of methadone as low as 20mg/day. Findings from multiple sensitivity analyses were similar to those from the primary analysis.

Our findings differed from those of a prior cohort study in VA patients, which reported that adjusted overall mortality for methadone users was 44% lower than that for morphine SR users.18 This unexpected finding might be partially explained by the study design. Because the VA study included patients with life-threatening illnesses, the results could reflect imbalances between the two study opioid groups incompletely controlled for in the analysis. There was some evidence of imbalances; for example, 15% of methadone users had a diagnosis of cancer versus 26% of morphine users.

Confounding also is a potential explanation for the increased mortality we observed in methadone patients. Such bias would occur if study methadone users had had greater risk of death related to either baseline comorbidity or to the indications for and characteristics of opioid use. However, there were several lines of evidence that our findings are not due to confounding.

The cohort excluded opioid users most likely to introduce confounding: those with elevated risk of death unrelated to therapeutic opioid use. Thus, we excluded patients with cancer and other diagnosed serious illnesses. Consequently, 84% of deaths in the study cohort were potentially related to opioid toxicity. Morphine SR and methadone patients were comparable with regard to indications for the study opioids, their prescribed doses, and the use of cardiovascular, psychotropic, and other musculoskeletal medications. The HRs were adjusted for differences between the groups in a statistical analysis that controlled for 196 potential confounders. Finally, results did not differ materially in a propensity-score-matched analysis in which the absolute difference in covariate prevalence between the two groups was never greater than 1%.

In the absence of confounding or bias, the occurrence of deaths less likely to be related to opioid toxicity should not differ according to study opioid use. The HR for these deaths was 1.26 (0.70–2.26), which was less than that for other study deaths and was not statistically significant. However, given the study design, only 16% of deaths fell into this category; thus, an increased risk could not be ruled out.

Nearly three-fourths of the study deaths met accepted definitions for either opioid overdose death or sudden cardiac death. For these deaths, the risk was most pronounced for the opioid overdose deaths, regardless of whether or not they also met the definition for sudden cardiac death. Indeed, for such deaths, the risk for methadone users was more than twice that of morphine SR users. In contrast, for deaths only meeting the definition for sudden cardiac death, the small increase in risk for methadone users was not statistically significant. This finding persisted in analyses with mutually exclusive definitions for opioid overdose and sudden cardiac death and restricted to adjudicated deaths.

The absence of a significantly increased risk for deaths only meeting the definition for sudden cardiac death may reflect the relatively low methadone doses received by the study cohort. The proarrhythmic effects of methadone are strongly dose-dependent and have been most frequently reported in patients receiving the higher doses prescribed for treatment of opioid addiction.10 Most arrhythmia case reports involve doses greater than 100mg/day,11;12 a threshold for increased electrocardiographic monitoring of patients receiving methadone.10 In this cohort of patients treated for non-cancer pain, the median methadone dose was 40mg/day; only 11% of patients had doses exceeding 100mg/day.

This finding also could be influenced by misclassification of arrhythmia-related deaths as due to opioid overdose. The clinical circumstances of each type of death may be similar: for example, an unexpected death during sleep. Post-mortem drug levels often cannot reliably identify unintentional overdose deaths, given the highly patient-specific opioid toxicity threshold and the changes in drug levels after death.29;39 When confronted with a sudden unexpected death in a chronic opioid user, a medical examiner reasonably might classify it as overdose-related. Even when elevated drug levels are present, the proximate cause of death could be a ventricular arrhythmia. Indeed, it is widely acknowledged that given the generally available postmortem data, it is difficult to distinguish unexpected deaths due to respiratory depression from those due to arrhythmias.11;13

Methadone users had increased risk of death during followup for doses as low as 20mg, which is consistent with the complex pharmacology of methadone. Because the duration of respiratory depression is greater than that for analgesia, repeated use of low doses to control pain may lead to drug accumulation and inadvertent overdose.6 The likelihood of drug accumulation may be increased by methadone's highly variable elimination half life, ranging from 7–65 hours.4;5

Our study had several limitations. Approximately one-fourth of the cohort received study opioids prior to cohort entry, which could attenuate risk due to loss of persons particularly susceptible to methadone's adverse effects ("depletion of susceptibles"). However, restricting the cohort to new users did not materially change study findings. More than one-half of the cohort had concurrent use of a non-study opioid, reflecting the common practice of prescribing short-acting opioids for breakthrough pain. The drug abuse exclusion was based on medical care encounters, which could lead to bias if methadone was prescribed for unrecorded substance abuse. However, the increased risk for the lowest doses of methadone suggests this did not explain study findings. The cohort consisted of Tennessee Medicaid enrollees and thus both program and population characteristics might affect study findings.

In conclusion, in patients with non-cancer pain, the risk of out-of-hospital death in methadone users was 46% greater than that for users of morphine SR. The absolute excess risk was 72 deaths per 10,000 person-years of study opioid use. Increased risk was present for methadone doses as low as 20mg/day. These findings support recommendations that methadone should not be considered a drug of first choice for pain.

Acknowledgments

Supported by a grant from the NHLBI (# HL081707), the NIAMS (#K23AR064768) and a Vanderbilt Physician Scientist Development award. We gratefully acknowledge the Tennessee Bureau of TennCare, the Tennessee Department of Health and Tennessee Donor Services, which provided study data.

Appendix

This appendix provides additional details for the methadone study and should be read in conjunction with the primary manuscript.

1. Cohort

The cohort included all Medicaid enrollees with at least one prescription for methadone or morphine SR during the study period such that, on the day the prescription was filled (t0), the patient met study inclusion/exclusion criteria (Appendix Table 1).

Criteria 1–4 are designed to identify a population in which the occurrence of sudden unexpected death should be infrequent. See the primary manuscript for the rationale for each criterion. We required cohort members to be at least 30 years of age, given the lower prevalence of chronic pain in younger persons.

Criterion 5 is designed to exclude persons with recorded evidence of drug abuse. The purpose was to exclude both patients receiving methadone for treatment of opioid dependence and persons whose overdose deaths might be related to non-medical opioid use. Any recorded evidence of drug abuse triggered the criteria, including:

  1. Diagnoses indicating drug abuse: These include ICD9-CM codes of 292.0 (drug withdrawal syndrome), 304.x (drug dependence), 305.2–305.9 (drug abuse, except alcohol/tobacco), 965.01 (accidental poisoning, heroin), 969.6 (poisoning, psychodysleptic [hallucinogens]), 970.81 (cocaine poisoning), E8500 (heroin poisoning), E8541 (psychodysleptic poisoning);

  2. Diagnoses indicating inpatient treatment for alcohol use disorder: Any hospitalization with a primary discharge diagnosis of 291.x, 303.x, 305.0, 980.0, 980.9, E860.0, E860.1, E860.9. Patients with such severe alcohol use disorders may be at increased risk for overdose related to non-medical opioid use;

  3. Procedures: Any procedure indicating treatment of drug addiction, excluding alcohol addiction treatment (unless this is inpatient).

  4. Medications. Any filled prescriptions for buprenorphine, as its primary use is for drug addiction.

Criteria 6 and 7 are related to the availability in the Medicaid files of the medical encounters needed to define exposure to study opioids and comorbidity. Thus, we required that cohort members have Medicaid enrollment with pharmacy benefits for at least one year as well medical care utilization during that year. Given that most study covariates were ascertained from medical care encounters, this assured some degree of medical surveillance.

Criterion 8 does not allow a filled prescription for the other study opioid in the past 30 days because of potentially overlapping use.

Appendix Table 1.

Cohort inclusion/exclusion criteria; t0 is the date of the prescription fill. Numbers are the number of subjects with any use of the study opioids during the study period who met the specific inclusion/exclusion criteria.

Criterion Description Morphine SR, N Methadone, N
1. Age Age 30–74 years at t0. 45,037 12,248
2. Cancer or other serious illness No evidence of illness on t0 or the preceding 365 days for which an out-of-hospital death might be expected. Exclusion diseases were cancer, HIV, renal/liver/cardio-respiratory failure, organ transplant, degenerative musculoskeletal disorders (e.g., multiple sclerosis), potentially lethal congenital anomalies or childhood conditions, or other evidence of end-stage illness. 44,112 12,141
3. Institution Not residing in a nursing home or other residential institution on t0 or at any time in the preceding 365 days, except for stays of <30 days following hospital discharge. 41,662 11,531
4. Recent hospitalization Not in the hospital on t0 or the preceding 29 days. 39,258 10,458
5. Drug abuse No recorded evidence of drug abuse (except for alcohol/tobacco) on t0 or the preceding 365 days. 35,374 9,370
6. Enrollment Enrolled with full pharmacy benefits on t0 and the preceding 365 days. 34,748 9,133
7. Medical care At least one filled prescription as well as two encounters with a diagnosis in the 365 days preceding t0. 34,540 8,487
8. Other opioid
  1. Morphine SR: No prescription for methadone filled on t0 or in past 30 days;

  2. Methadone: No prescription for morphine SR filled on t0 or in past 30 days.

32,742 6,014

2. Followup

The patient entered the cohort on the date of filling of the first prescription for a study opioid during the study period that met the criteria in Appendix Table 1.

Followup consisted of periods of current opioid use that met the study inclusion/exclusion criteria. These periods were identified from filled prescriptions for study opioids. The duration of current use for the prescription was identified from the dispensed days of supply, edited to resolve infrequent inconsistencies with quantity dispensed. In Tennessee Medicaid, filled prescriptions during the study period almost always were limited to 30 days of supply.

To define study followup, we evaluated all study opioid prescriptions filled during the study period. Those that met the inclusion/exclusion criteria contributed to current use person-time.

If a person changed to the other study opioid during followup, we excluded the 30 days following the end of the last prescription for the prior opioid from followup. This "wash out" period reduced the potential for exposure misclassification, given the potential overlap in study opioid use. Subsequent current use person-time accrued to the second opioid.

Study followup ended with the last day of the study period, the last day of supply of the last qualifying prescription, irreversible failure to meet the inclusion/exclusion criteria (e.g., age 75+), or death. Appendix Figure 1 depicts cohort followup for 4 hypothetical persons. Person a has 5 qualifying prescriptions for morphine SR, with a short interval between the end of the days of supply for prescription 2 and the filling of prescription 3, as indicated in gray. This period is excluded from study followup to reduce misclassification; it is unclear whether it represents current use or non-use. All current use person-time defined by the 5 prescriptions goes into the morphine SR category. Person b has a similar pattern of use of methadone. Person c has 2 qualifying prescriptions for morphine SR, which contribute to morphine SR person-time. After the second prescription, person 2 switches to methadone. The first prescription for methadone, indicated by a lighter color, is the "wash out" period and is not included in followup. However, the next 2 qualifying prescriptions (3 and 4) contribute person-time to the methadone current use category. Person d shows a similar switch from methadone to morphine SR.

graphic file with name nihms641066f3.jpg

Appendix Figure 1. Study followup for four hypothetical cohort members (persons a, b, c, d). Numbers in bars represent qualifying study opioid prescriptions.

3. Endpoint Classification

Deaths during followup were identified from the linked death certificate-Medicaid enrollment file. These were further classified to provide insight into the potential mechanisms for opioid toxicity.

a. Sudden unexpected deaths

The primary mechanisms by which opioids are thought to increase the risk of death are respiratory depression1 and cardiac arrhythmias.26 Thus, the deaths most likely to be related to opioid toxicity were those meeting previous definitions for either an opioid overdose death1;7 or a sudden cardiac death.2;810 We termed these sudden unexpected deaths because they are unexpected and typically occur within a short interval in persons in a usual state of health who have no evident cause for the death other than opioid toxicity.

b. Opioid overdose deaths

We based this definition on the underlying cause of death code1 because previous experience indicates this diagnosis reliably identifies opioid overdose deaths. A comparison of death certificate diagnoses of overdose deaths with medical examiner data reported a sensitivity of 95% and a positive predictive value of 94%.11 Thus, opioid overdose deaths had a death certificate underlying cause of death code indicating unintentional or intentional (including intent undetermined) opioid overdose (Appendix Table 2).

We considered refining the definition for opioid overdose deaths by incorporating information on drug levels when this was available. However, post-mortem drug levels often cannot reliably identify unintentional overdose deaths, given the highly patient-specific opioid toxicity threshold and the changes in drug levels that occur after death.12;13

We included deaths with an unidentified responsible drug because a large proportion of such deaths are related to opioid overdose, particularly those involving multiple drugs.14 We included the small number of deaths coded as due to an intentional overdose because suicide attempts with methadone might be more likely to be successful than those with morphine SR.

Appendix Table 2.

Cause of death codes consistent with opioid overdose death.

ICD-9 ICD-10
Adverse effects of medications
E935.0 Y45.0 Adverse effects, opioids
E935.8 Adverse effects, pentazocine
E935.9 Y45.9 Adverse effects, unspecified analgesic
E940.1 Y50.1 Adverse effects, opioid antagonist
E947.9 Y57.9 Adverse effects, unspecified medication
Medication poisoning/overdose, unintentional
E850.0 X42 Accidental poisoning, narcotics
E850.8 Accidental poisoning, pentazocine
E850.9 Accidental poisoning, unspecified analgesic
E854.3 Accidental poisoning, opioid agonist
E858.9 X44 Accidental poisoning, unspecified medication
Medication poisoning/overdose, intent undetermined
E980.0 Y12 Undetermined intent poisoning, narcotics
E980.5 Y14 Undetermined intent poisoning, unspecified medication
Medication poisoning/overdose, intentional
E950.0 X62 Intentional self poisoning, narcotics
E950.5 X64 Intentional self poisoning, unspecified medication

c. Sudden cardiac deaths: definition

Sudden cardiac deaths were those with documentation of death within one hour of symptom onset or that the patient was alive and in the usual state of health within 24 hours of death with no plausible non-cardiac cause of death.810 This definition excluded deaths from arrests in a hospital or other institutional setting, with documentation suggesting an underlying noncardiac cause (e.g., pneumonia) or a different cardiac etiology (e.g., heart failure or bradyarrhythmia). However, given the potential similarity of the clinical circumstances of sudden cardiac and opioid overdose deaths (see §3.e below), sudden cardiac death was not ruled out for deaths coded as due to overdose.

d. Sudden cardiac deaths: identification

To identify sudden cardiac deaths, we reviewed and adjudicated terminal medical records, when available, for all deaths during followup. The records included autopsy reports, emergency medical services records, emergency department visit records, organ donation service records, and hard copies of the death certificates. Each death was independently adjudicated by two investigators. If there was disagreement, the case was adjudicated at a regular investigators meeting.

If terminal medical records were unavailable, we identified sudden cardiac deaths from a computer definition based on both death certificate diagnosis and terminal medical care encounters. This definition had an estimated positive predicted value of 87%-90%.8 However, the estimated sensitivity of this definition was less than 75% because it was restricted to underlying cause of death codes with a positive predictive value greater than 70%.8 Thus, the computer definition was used only when medical records were unavailable.

e. Overlap between opioid overdose and sudden cardiac deaths

In chronic opioid users, the clinical circumstances of an overdose and sudden cardiac death may be similar (e.g. an unexpected death during sleep) and it often is difficult to distinguish these mechanisms post mortem.3;5 Thus, when terminal medical records were available, a death with an underlying cause of death code indicating opioid overdose also could be adjudicated as meeting the sudden cardiac death definition. For example, a patient who died during sleep with no compelling evidence of an overdose in the terminal medical records would have been adjudicated as consistent with sudden cardiac death. However, such overlap could not occur for the computer definition because overdose cause of death codes were not part of the case definition.8

f. Other respiratory/cardiovascular deaths

Other respiratory/cardiovascular deaths were those for which opioid involvement was possible but less certain. This category included deaths with an underlying cause of death indicating chronic obstructive pulmonary disease or a bacterial or viral respiratory infection. This category also included unadjudicated deaths with an underlying cardiovascular cause of death that did not meet our computer case definition8 for sudden cardiac death. These were included because our previous validation study8 found that such deaths, when adjudicated, do include sudden cardiac deaths, although the PPV was less than 70%, the threshold for inclusion in the computer case definition. This group included deaths coded as due to diabetes, as the terminal event often is cardiac8 as well as those with an ill-defined cause, given that in the study population cardiovascular causes are a predominant cause of death.

g. Other deaths

Other deaths were all other deaths during followup, which, given that they did not meet the definition for the above categories, were considered as less likely to be related to opioid toxicity. These were primarily injury deaths, cardiovascular deaths for which sudden cardiac death was deemed unlikely following medical record adjudication, alcohol-related deaths, and deaths due to non-respiratory infections.

h. Classification of deaths

There were 477 deaths during study followup (Appendix Figure 2). Medical records were available for 300 deaths. These included 229 sudden unexpected deaths, 8 other respiratory/cardiovascular deaths, and 63 other deaths. The sudden unexpected deaths included 10 deaths meeting the definition for opioid overdose alone, 144 deaths meeting the definition for sudden cardiac death alone, and 75 deaths meeting the definition for both opioid overdose and sudden cardiac death2.

Of the 177 deaths for which terminal medical records were not available, 117 were sudden unexpected deaths, 45 other respiratory/cardiovascular deaths, and 15 other deaths. The sudden unexpected deaths included 33 with an underlying cause of death of opioid overdose and 84 that met the computer case definition for sudden cardiac deaths. Because the computer case definition did not include deaths coded as due to overdose, there was no overlap.

graphic file with name nihms641066f4.jpg

Appendix Figure 2. Study death adjudication. CV = cardiovascular, SCD = sudden cardiac death, OD = overdose.

4. Study Opioid Indication

One of the covariates included in our analyses was the indication for the study opioids. Both study opioids are nearly always prescribed for chronic pain. Thus, the algorithm for assigning indication gave priority to chronic indications and did not change a chronic indication until there was a gap of at least 365 days between filled study opioid prescriptions.

For a given cohort member, we began with the earliest prescription for a study opioid. We looked for an indication from diagnoses for outpatient medical care encounters. We gave priority to temporal proximity to the prescription fill date (most important), type of medical care (next), and the specific indication.

The algorithm considered the date the prescription was filled, t0, and each of the 7 preceding days. The algorithm first sought an indication from medical care on t0, it next considered t0-1, etc.

The types of medical care considered were primary/secondary diagnoses and procedures of physician and other outpatient, ED, inpatient encounters. The priority was inpatient first, then ED, then outpatient. Within each setting, the priority was procedure, then primary diagnosis, then secondary diagnosis.

For the indications, priority was given to chronic indications, then to acute indications. Within these groups, the priority was as specified below (based on our assessment of the relative frequency of the conditions).

For chronic conditions, the priorities were:

  1. Back pain/degenerative back disorders;

  2. Other musculoskeletal/soft tissue pain;

  3. Abdominal pain;

  4. Headache, including migraine;

  5. Other neurological pain.

For acute conditions, the priorities were:

  1. Infection;

  2. Trauma: fracture, sprain, contusion, wound, other/unspecified trauma;

  3. Outpatient surgery/procedure ;

  4. Dental or other oral procedure.

5. Statistical Analysis

Given the large number of study covariates, we controlled for these by stratifying the regression analyses by deciles of either a propensity score1517 or a disease risk score.17;18 This analysis used SAS PROC PHREG with the STRATA statement. This produces maximum partial likelihood estimates of a common parameter across strata (Allison19 p. 180).

The propensity score was estimated from a logistic regression model with the prescription as the unit of analysis. Appendix Table 3 shows the distribution of patient covariates included in the propensity score. The distribution is according to the time of the filling of each qualifying cohort prescription.

Appendix Table 3.

Distribution of variables in the propensity score according to study opioid use. Unless otherwise stated, variables are binary and table values are proportions.

Morphine SR Methadone
1. Age at baseline, years 48.37 47.78
2. Sex female 57.87% 57.85%
3. White race 84.35% 84.58%
4. Standard Metropolitan Statistical Area 47.18% 55.15%
5. Medicaid enrollment disabled 69.65% 68.34%
6. Medicaid enrollment uninsured 14.51% 15.94%
7. <=1997 1.35% 0.55%
8. 1998 1.31% 0.70%
9. 1999 2.70% 1.65%
10. 2000 4.84% 3.64%
11. 2001 6.89% 6.82%
12. 2002 8.64% 12.49%
13. 2003 11.97% 16.53%
14. 2004 17.71% 20.11%
15. 2005 17.95% 20.15%
16. 2006 7.35% 5.83%
17. 2007 7.04% 5.46%
18. 2008 6.69% 4.27%
19. 2009 5.57% 1.80%
20. January 8.07% 8.16%
21. February 7.51% 7.47%
22. March 8.44% 8.34%
23. April 8.30% 8.34%
24. May 8.22% 8.26%
25. June 8.17% 8.01%
26. July 8.63% 8.66%
27. August 8.39% 8.48%
28. September 8.00% 8.17%
29. October 8.53% 8.71%
30. November 8.52% 8.51%
31. December 9.22% 8.90%
32. Back pain 48.12% 49.15%
33. Other musculoskeletal pain 25.42% 21.28%
34. Abdominal pain 3.94% 3.91%
35. Headache 1.95% 2.80%
36. Other neurologic condition 2.17% 3.06%
37. Infection 2.59% 2.11%
38. Trauma 4.23% 4.02%
39. Surgery or procedure 7.81% 8.64%
40. Dental 0.51% 0.62%
41. No indication found 3.27% 4.43%
42. Non-study opioid on t0 65.21% 47.61%
43. High dose non-study opioid t0 9.14% 9.57%
44. Parenteral opioid on t0 or prior day 0.61% 0.35%
45. Benzodiazepine, former user 13.06% 11.55%
46. Benzodiazepine, indet user 9.32% 9.60%
47. Benzodiazepine, current, <5mg diazepam 2.43% 2.84%
48. Benzodiazepine, current, 6–9mg 7.09% 7.36%
49. Benzodiazepine, current, 10–19mg 13.27% 14.83%
50. Benzodiazepine, current, 20+mg 18.20% 18.09%
51. Other GABA agonist 18.97% 19.89%
52. TCA, former user 9.59% 9.64%
53. TCA, indet user 4.35% 4.33%
54. TCA, current, <100 mg amitriptyline 6.73% 6.53%
55. TCA, current, 100–299 mg 5.89% 5.18%
56. Selective serotonin reuptake inhibitor 53.20% 56.50%
57. Trazodone 16.27% 17.65%
58. Other antidepressant 18.80% 20.87%
59. Antipsychotic, former user 5.00% 5.67%
60. Antipsychotic, indet user 2.83% 3.29%
61. Antipsychotic, current, <100 mg chlorpromazine 1.20% 1.28%
62. Antipsychotic, current, 100–199mg 2.41% 2.80%
63. Antipsychotic, current, 200–299mg 1.74% 2.22%
64. Antipsychotic, current, 300mg+ 2.73% 2.81%
65. Anticonvulsant mood stabilizer 37.85% 39.74%
66. Lithium 1.26% 0.92%
67. Hydroxyzine 15.74% 15.01%
68. Other anxiolytic 0.27% 0.32%
69. Psychostimulant 2.35% 4.27%
70. Schizophrenia 1.30% 1.17%
71. Paranoia 0.08% 0.11%
72. Depression 10.37% 11.92%
73. Depression, severe 8.60% 9.98%
74. Bipolar 5.92% 6.46%
75. Other affective disorder 14.57% 13.96%
76. Anxiety disorder 11.38% 10.26%
77. Panic disorder 2.63% 3.34%
78. Borderline personality disorder 0.18% 0.24%
79. Organic mental illness 0.55% 0.55%
80. Adjustment disorder 3.33% 3.36%
81. Other mental symptoms 7.03% 8.25%
82. Attempted self harm 0.58% 0.61%
83. Alcohol abuse 2.00% 1.82%
84. ACE inhibitor 30.63% 28.28%
85. Angiotensin receptor blocker 8.13% 8.02%
86. Anticoagulant 5.82% 4.25%
87. Antiarrhythmic 2.10% 2.41%
88. Aspirin 6.01% 5.84%
89. Beta blocker 23.36% 21.33%
90. Calcium channel blocker 20.47% 17.96%
91. Digoxin 2.28% 1.55%
92. Loop diuretic 21.75% 22.34%
93. Other diuretic 23.80% 24.97%
94. Insulin 8.58% 8.37%
95. Oral hypoglycemic 15.52% 15.34%
96. Statin 27.85% 25.46%
97. Fibrate 6.65% 6.95%
98. Nitrate 12.05% 10.19%
99. Other antihypertensive 8.36% 8.22%
100. Peripheral vasodilator 1.71% 1.58%
101. Platelet inhibitor 6.49% 5.30%
102. Angina 5.59% 4.49%
103. Coronary revascularization 1.24% 1.04%
104. Myocardial infarction 0.89% 0.62%
105. Other coronary heart disease 12.40% 10.33%
106. Mitral valve disorder 2.06% 2.12%
107. Other valve disorder 1.26% 0.88%
108. Conduction disorder 0.73% 0.68%
109. Arrhythmia 4.41% 4.14%
110. Atrial fibrillation 1.40% 1.05%
111. Congestive heart failure 5.96% 4.94%
112. Stroke 2.09% 1.92%
113. TIA 1.63% 1.51%
114. Other cerebrovascular 1.58% 1.44%
115. Peripheral vascular disease 4.80% 4.21%
116. Obesity 3.05% 2.86%
117. Morbid_obesity 2.07% 1.62%
118. Smoking 20.92% 17.32%
119. Hypertension 43.68% 40.60%
120. Malignant hypertension 2.23% 1.94%
121. Hyperlipidemia 21.98% 19.58%
122. Chronic renal failure 1.12% 1.08%
123. Renal insufficiency 1.42% 1.19%
124. Diabetes 21.25% 19.76%
125. Other cardiovascular 16.00% 14.31%
126. Diabetes, ocular complications 1.86% 1.87%
127. Diabetes, neurologic complications 3.76% 4.34%
128. Diabetes, skin complications 1.98% 2.40%
129. Diabetes, other complications 4.06% 4.42%
130. Diabetes, poor control 7.36% 7.49%
131. New cardiovascular diagnosis 6.44% 5.98%
132. New cardiovascular medication 4.93% 4.74%
133. Anticonvulsant 12.87% 16.78%
134. Antiparkinsonion medication 3.77% 3.74%
135. Seizure disorder 5.16% 4.79%
136. Parkinsons disease 0.28% 0.40%
137. Other movement disorder 2.24% 2.39%
138. Alzheimers dementia 1.06% 1.06%
139. Rheumatoid arthritis, DMARD 4.69% 3.71%
140. NSAID/coxib 62.69% 63.01%
141. Musculoskeletal relaxant 65.00% 64.82%
142. Other analgesic 5.87% 5.27%
143. Corticosteroid 5.92% 5.07%
144. Inflammatory arthropathy 10.89% 9.71%
145. Osteoarthritis 22.78% 19.99%
146. Other immunologic disease 1.95% 1.73%
147. Injury 42.24% 40.45%
148. Unintentional fall 9.59% 9.14%
149. Wheelchair or walker 7.35% 6.44%
150. Incontinence 2.53% 2.67%
151. Other frailty 4.09% 4.26%
152. Beta agonist 33.85% 30.70%
153. Other bronchodilator 20.02% 17.89%
154. COPD 23.74% 18.49%
155. Asthma 11.14% 10.73%
156. Respiratory infection 51.16% 48.17%
157. CPAP 3.52% 3.15%
158. Home oxygen 8.88% 6.45%
159. Antibiotic 81.00% 79.97%
160. Hormone replacement therapy 21.47% 23.59%
161. GI drugs 59.02% 57.60%
162. Oral contraceptive 1.87% 2.30%
163. Pregnancy 0.47% 0.41%
164. TDP drug, former 23.64% 23.63%
165. TDP drug, indet 12.42% 12.09%
166. TDP drug, current 2.07% 1.83%
167. CV hospitalization, [t0–365,t0–91] 3.61% 3.11%
168. CV hospitalization, [t0–90,t0–30] 0.90% 0.78%
169. CV hospitalization, 5+ days past year 1.61% 1.45%
170. Other hospitalization past year 14.97% 14.09%
171. Psychiatric inpatient 8.78% 8.18%
172. CV ED visit, past year 21.89% 19.82%
173. Non_CV ED visit, past year 57.09% 54.95%
174. CV ED, [t0–29,t0–8] 1.22% 1.16%
175. CV ED, [t0–7,t0–1] 0.40% 0.39%
176. Other ED, [t0–29,t0–8] 6.33% 5.62%
177. Other ED, [t0–7,t0–1] 2.60% 2.30%
178. CV outpatient, 1 past year 15.17% 15.92%
179. CV outpatient, 2–4 past year 31.04% 29.67%
180. CV outpatient, 5–9 past year 14.64% 13.80%
181. CV outpatient, 10+ past year 10.35% 9.00%
182. Other outpatient, 1 past year 0.50% 0.72%
183. Other outpatient, 2–4 past year 5.62% 5.89%
184. Other outpatient, 5–9 past year 12.97% 13.52%
185. Other outpatient, 10+ past year 80.68% 79.67%
186. Injury hospitalization past year 2.87% 2.57%
187. Injury ED, 1 past year 17.19% 17.55%
188. Injury ED, 2–5 past year 8.58% 7.67%
189. Injury ED, 6+ past year 0.73% 0.51%
190. Injury outpatient, 1 past year 17.80% 18.03%
191. Injury outpatient, 2–10 past year 17.75% 16.43%
192. Injury outpatient, 11+ past year 0.77% 0.80%
193. Poisoning/overdose ED/inpatient past year 3.38% 3.67%
194. Any outpatient, [t0–30,t0–1] 81.00% 79.77%
195. Any Rx fill, [t0–30,t0–1] 96.07% 96.13%
196. Home health visit-months past year 0.08 0.05

Appendix Table 4 shows key quantiles from the distribution of the propensity score for both the morphine SR and methadone study groups.

Appendix Table 4.

Distribution of propensity score according to study opioid.

Morphine SR Methadone
Propensity
Score Minimum 0.0093939 0.0162338
1st percentile 0.033038 0.0530837
5th percentile 0.0539115 0.0829634
10th percentile 0.070159 0.1028154
25th percentile 0.1015634 0.1437964
50th percentile 0.1470371 0.2111172
75th percentile 0.2132532 0.2937957
90th percentile 0.2887691 0.3656802
95th percentile 0.3350258 0.4071634
99th percentile 0.4192937 0.4862794
Maximum 0.7497266 0.6915911

The global mortality risk score was a disease risk score for risk of total study mortality. The disease risk score is often described as the prognostic analogue of the propensity score.20 It is the summary risk of the study endpoint as a function of the covariates, given the reference category for the exposure. Disease risk scores are more suitable than propensity scores for non-binary comparisons.17;18;21

In the present study, the incidence of death during the period of current use for a given prescription can be described as

  • I = L*exp(z'b)

where

  • I is the incidence of death, expressed as deaths per person-year

  • L is the length of the period of current use, expressed in years

  • z is the vector of covariates at the time of the prescription fill

  • b are the logs of the incidence rate ratio for each covariate.

We used Poisson regression to estimate exp(z'b), which in turn estimates the annual risk of death (when endpoints are infrequent), given the covariate values at the time of the prescription fill. The regression was performed for the entire cohort and then z'b was calculated, with the coefficient for methadone set to zero. Although it is possible to estimate the score in non-users of methadone, experience suggests that in the absence of effect modification, the estimate is better if the entire cohort is used.17

For descriptive purposes, the global mortality risk score was expressed as 20 quantiles. Appendix Figure 3 shows the annual incidence of study deaths according to these quantiles.

graphic file with name nihms641066f5.jpg

Appendix Figure 3. Annual incidence of deaths according to the global mortality risk score quantile. Each quantile represents approximately 5% of study followup person-time.

The analysis according to dose of study opioid therapy proceeded as follows. First, we defined a single categorical variable with 8 categories: 4 for each combination of study opioid and dose quartile. For example, the first 4 categories would be for morphine SR in doses of <60mg, 60–89mg, 90–159mg, and ≥160mg.

The annual incidence (and its 95% confidence interval) was estimated for each of the 8 categories. That for the reference category (morphine SR <60mg) was unadjusted. For the other categories the unadjusted annual incidence was multiplied by the HR (and its 95% confidence interval) to produce the adjusted annual incidence ( and its 95% confidence interval).

The parameters for testing hypotheses and for methadone:morphine SR comparisons were estimated as appropriate linear combinations. For example, the dose-response test for morphine is a linear combination with coefficients −3 −1 1 −3 0 0 0 0. The comparison for the lowest dose quartile for methadone vs that for morphine SR used the coefficients −1 0 0 0 1 0 0 0.

6. Additional Findings

a. Propensity-score matched cohort

The propensity scores were calculated at the time of each prescription fill. The distribution of the propensity score for methadone prescriptions was divided into centiles. From each centile, we randomly selected one morphine SR prescription for every methadone prescription in the centile.

Appendix Table 5 shows the characteristics of the propensity-score-matched cohort.

Appendix Table 5.

Propensity-score-matched cohort characteristics. See MS Table 1.

Morphine SR Methadone
Study opioid prescriptions during followup, N 76,035 76,035
Year of prescription, mean 2003.8 2003.8
Age, years, mean 47.8 47.8
Female 57.8% 57.8%
White 84.6% 84.5%
Medicaid enrollment disabled 68.6% 68.3%

Study opioid prescription characteristics
Opioid indication

   Back pain 77.0% 77.2%
   Other musculoskeletal pain 10.8% 10.7%
   Other chronic pain: soft tissue, abdominal, or neurologic 3.9% 3.9%
   Acute pain related to medical/dental procedure, trauma, or infection 1.5% 1.4%
   No indication recorded 6.8% 6.8%
Opioid supply dispensed, days 27.9 27.6
Dose per day, mg
   Quartile 1 60.0 20.0
   Median 90.0 40.0
   Quartile 3 150.0 60.0
   Greater than 100mg methadone or 200mg morphine SR 12.4% 11.4%
Study opioids, total prescribed days past year 219.2 222.1
Non-study opioids, current use 47.3% 47.7%

Psychotropic medications

Benzodiazepine 64.0% 64.2%
Cyclic antidepressant 25.8% 25.6%
SSRI or other antidepressant 67.0% 67.2%
Antipsychotic, % 18.1% 18.1%

Cardiovascular medications

Digoxin 1.6% 1.6%
Loop diuretic 22.5% 22.4%
Insulin or oral hypoglycemic 18.6% 18.1%

Musculoskeletal medications

NSAID/coxib 63.2% 63.0%
Musculoskeletal relaxant 64.7% 64.9%
Corticosteroid, systemic 4.9% 5.1%

Respiratory medications and diagnoses

Beta agonist 30.6% 30.7%
Other bronchodilator 17.8% 17.9%
COPD 18.8% 18.5%
Asthma 10.8% 10.8%
Home oxygen 6.5% 6.5%

Medical care utilization

Hospital stay past year 17.0% 16.9%
ED visit past 30 days 8.6% 8.5%
Overdose ED/inpatient past year 3.6% 3.6%

b. Hazard ratios according to adjudication status and type of death

Appendix Figure 4 shows the hazard ratios according to whether or not medical records were available for adjudication and by type of death.

graphic file with name nihms641066f6.jpg

Appendix Figure 4. Study death adjudication. CV = cardiovascular, SCD = sudden cardiac death, OD = overdose. Hazard ratios (HRs) are indicated in red with 95% confidence intervals in parentheses.

c. Mutually exclusive definitions for opioid overdose and sudden cardiac death

We conducted sensitivity analyses with mutually exclusive definitions for both opioid overdose and sudden cardiac death (Appendix Table 5). In one analysis, priority was given to opioid overdose; hence, deaths meeting the definition for both overdose and sudden cardiac death were considered opioid overdose deaths. In a second analysis, the deaths meeting both definitions were considered sudden cardiac deaths. We also performed these analyses with the deaths restricted to those which were adjudicated. In all of these analyses, the HRs for opioid overdose deaths were greater than those for sudden cardiac deaths.

Appendix Table 6.

Risk of opioid overdose or sudden cardiac death during current use of morphine sustained release (SR) and methadone, using mutually exclusive definitions to classify deaths. HR denotes adjusted hazard ratio; CI denotes confidence interval.

Morphine SR Methadone
Study followup

Person-years 23,609 5,091

Deaths

Rate/10,000 person-years (N deaths) HR (95% CI) p-value
All Deaths
Priority given to opioid overdose
   Opioid overdose death 33.5 (79) 76.6 (39) 2.20 (1.48–3.29) .0001
   Sudden cardiac death 78.8 (186) 82.5 (42) 1.12 (0.80–1.59) .5067
Priority given to sudden cardiac death
   Opioid overdose death 11.4 (27) 31.4 (16) 2.54 (1.33–4.84) .0046
   Sudden cardiac death 100.8 (238) 127.7 (65) 1.33 (1.00–1.77) .0486
Adjudicated Deaths Only
Priority given to opioid overdose
   Opioid overdose death 24.6 (58) 53.0 (27) 2.16 (1.34–3.47) .0015
   Sudden cardiac death 50.4 (119) 49.1 (25) 1.00 (0.64–1.57) .9843
Priority given to sudden cardiac death
   Opioid overdose death 2.5 (6) 7.9 (4) 3.38 (0.91–12.47) .0608
   Sudden cardiac death 72.4 (171) 94.3 (48) 1.32 (0.95–1.84) .0995
1

The multiple cause of death data were not available for the entire study period.

2

In our review of medical records for the 85 deaths coded as due to opioid overdose, only one had evidence of another non-cardiac cause: a small pulmonary embolus found on autopsy which could have contributed to the death.

Reference List

  • 1.Boyer EW. Management of opioid analgesic overdose. N Engl J Med. 2012;367:146–155. doi: 10.1056/NEJMra1202561. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Krantz MJ, Martin J, Stimmel B, Mehta D, Haigney MCP. QTc interval screening in methadone treatment. Ann Intern Med. 2009;150:387–395. doi: 10.7326/0003-4819-150-6-200903170-00103. [DOI] [PubMed] [Google Scholar]
  • 3.Kao D, Bartelson BB, Khatri V, et al. Trends in reporting methadone-associated cardiac arrhythmia, 1997–2011. An analysis of registry data. Ann Intern Med. 2013;158:735–740. doi: 10.7326/0003-4819-158-10-201305210-00008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Pearson EC, Woosley RL. QT prolongation and torsades de pointes among methadone users: reports to the FDA spontaneous reporting system. Pharmacol Drug Safety. 2005;14:747–753. doi: 10.1002/pds.1112. [DOI] [PubMed] [Google Scholar]
  • 5.Stringer J, Welsh C, Tommasello A. Methadone-associated Q-T interval prolongation and torsades de pointes. Am J Health-Syst Pharm. 2009;66:825–833. doi: 10.2146/ajhp070392. [DOI] [PubMed] [Google Scholar]
  • 6.Chugh SS, Socoteanau C, Reinier K, Waltz J, Jui J, Gunson K. A community-based evaluation of sudden death associated with therapeutic levels of methadone. Am J Med. 2008;121:66–71. doi: 10.1016/j.amjmed.2007.10.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Paulozzi LJ, Mack KA, Jones CM. Vital signs: Risk for overdose from methadone used as pain relief--United States, 1999–2010. Morbidty and Mortality Weekly Report. 2012;61:493–497. [PubMed] [Google Scholar]
  • 8.Chung CP, Murray KT, Stein CM, Hall K, Ray WA. A computer case definition for sudden cardiac death. Pharmacoepidemiol Drug Saf. 2010;19:563–572. doi: 10.1002/pds.1888. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Kawai VK, Murray KT, Stein CM, et al. Validation of a computer case definition for sudden cardiac death in opioid users. BMC Research Notes. 2012;5:473. doi: 10.1186/1756-0500-5-473. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Ray WA, Meredith S, Thapa PB, Meador KG, Hall K, Murray KT. Antipsychotics and the risk of sudden cardiac death. Arch Gen Psychiatry. 2001;58:1161–1167. doi: 10.1001/archpsyc.58.12.1161. [DOI] [PubMed] [Google Scholar]
  • 11.Landen MG, Castle S, Nolte KB, et al. Methodological issues in the surveillance of poisoning, illicit drug overdose, and heroin overdose deaths in New Mexico. Am J Epidemiol. 2003;157:273–278. doi: 10.1093/aje/kwf196. [DOI] [PubMed] [Google Scholar]
  • 12.Letsky MC, Zumwalt RE, Seifert SA, Benson BE. Cause of death conundrum with methadone use. A case report. Am J Forensic Med Pathol. 2011;32:193–196. doi: 10.1097/PAF.0b013e3181d3de94. [DOI] [PubMed] [Google Scholar]
  • 13.Hakkinen M, Launiainen T, Vuori E, Ojanpera I. Comparison of fatal poisonings by prescription opioids. Forensic Sci Int. 2012;222:327–331. doi: 10.1016/j.forsciint.2012.07.011. [DOI] [PubMed] [Google Scholar]
  • 14.Paulozzi LJ, Xi Y. Recent changes in drug poisoning mortality in the United States by urban-rural status and by drug type. Pharmacoepidemiol Drug Saf. 2008;17:997–1005. doi: 10.1002/pds.1626. [DOI] [PubMed] [Google Scholar]
  • 15.Shadish WR, Steiner PM. A primer on propensity score analysis. Newborn & Infant Nursing Reviews. 2010;10:19–26. [Google Scholar]
  • 16.Austin PC. An introduction to propensity score methods for reducing the effects of confounding on observational studies. Multivariate Behav Res. 2011;46:399–424. doi: 10.1080/00273171.2011.568786. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Arbogast PG, Ray WA. Performance of disease risk scores, propensity scores, and traditional multivariable outcome regression in the presence of multiple confounders. Am J Epi. 2011;174:613–620. doi: 10.1093/aje/kwr143. [DOI] [PubMed] [Google Scholar]
  • 18.Arbogast PG, Ray WA. Use of disease risk scores in pharmacoepidemiologic studies. Statistical Meth Med Res. 2009;18:67–80. doi: 10.1177/0962280208092347. [DOI] [PubMed] [Google Scholar]
  • 19.Allison PD. Survival Analysis Using SAS. A Practical Guide. 2nd ed. Cary, NC: SAS Institute; 2010. [Google Scholar]
  • 20.Hansen BB. The prognostic analogue of the propensity score. Biometrika. 2008;95:481–488. [Google Scholar]
  • 21.Arbogast PG, Kaltenbach L, Ding H, Ray WA. Adjustment of multiple cardiovascular risk factors with a summary risk score. Epidemiol. 2008;19:30–37. doi: 10.1097/EDE.0b013e31815be000. [DOI] [PubMed] [Google Scholar]

References

  • 1.Paulozzi LJ, Mack KA, Jones CM. Vital signs: Risk for overdose from methadone used as pain relief--United States, 1999–2010. Morbidty and Mortality Weekly Report. 2012;61:493–497. [PubMed] [Google Scholar]
  • 2.Leppert W. The role of methadone in cancer pain treatment - a review. Int J Clin Pract. 2009;63:1095–1109. doi: 10.1111/j.1742-1241.2008.01990.x. [DOI] [PubMed] [Google Scholar]
  • 3.Haroutiunian S, McNicol ED, Lipman AG. Methadone for chronic non-cancer pain in adults. Cochrane database of systematic reviews. 2012:11. doi: 10.1002/14651858.CD008025.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Fredheim OM, Borchgrevink PC, Kaasa S, Dale O. Clinical pharmacology of methadone for pain. Acta Anaesthesiologica Scandinavica. 2008;52:879–889. doi: 10.1111/j.1399-6576.2008.01597.x. [DOI] [PubMed] [Google Scholar]
  • 5.Cebert Pharmaceuticals I. Methadone HCL. Columbus, Ohio: Boehringer Ingelheim Roxane, Inc; 2007. [Google Scholar]
  • 6.FDA. Drugs. Public Health Advisory: Methadone use for pain control may result in death and life-threatening changes in breathing and heart beat. U S Food and Drug Administration. 2006 [Google Scholar]
  • 7.Kuehn BM. Methadone overdose deaths rise with increased prescribing for pain. JAMA. 2012;308:749–750. doi: 10.1001/jama.2012.9289. [DOI] [PubMed] [Google Scholar]
  • 8.Pilgrim J, McDonough M, Drummer OH. A review of methadone deaths between 2001 and 2005 in Victoria, Australia. Forensic Sci Int. 2013;226:216–222. doi: 10.1016/j.forsciint.2013.01.028. [DOI] [PubMed] [Google Scholar]
  • 9.Bell JR, Butler B, Lawrence A, Batey R, Salmelainen P. Comparing overdose mortality associated with methadone and buprenorphine treatment. Drug Alcohol Depend. 2009;104:73–77. doi: 10.1016/j.drugalcdep.2009.03.020. [DOI] [PubMed] [Google Scholar]
  • 10.Krantz MJ, Martin J, Stimmel B, Mehta D, Haigney MCP. QTc interval screening in methadone treatment. Ann Intern Med. 2009;150:387–395. doi: 10.7326/0003-4819-150-6-200903170-00103. [DOI] [PubMed] [Google Scholar]
  • 11.Kao D, Bartelson BB, Khatri V, et al. Trends in reporting methadone-associated cardiac arrhythmia, 1997–2011. An analysis of registry data. Ann Intern Med. 2013;158:735–740. doi: 10.7326/0003-4819-158-10-201305210-00008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Pearson EC, Woosley RL. QT prolongation and torsades de pointes among methadone users: reports to the FDA spontaneous reporting system. Pharmacol Drug Safety. 2005;14:747–753. doi: 10.1002/pds.1112. [DOI] [PubMed] [Google Scholar]
  • 13.Stringer J, Welsh C, Tommasello A. Methadone-associated Q-T interval prolongation and torsades de pointes. Am J Health-Syst Pharm. 2009;66:825–833. doi: 10.2146/ajhp070392. [DOI] [PubMed] [Google Scholar]
  • 14.Marcus FI, Cobb LA, Edwards JE, et al. Mechanism of death and prevalence of myocardial ischemic symptoms in the terminal event after acute myocardial infarction. Am J Cardiol. 1988;61:8–15. doi: 10.1016/0002-9149(88)91295-7. [DOI] [PubMed] [Google Scholar]
  • 15.Hinkle LE, Thaler HT. Clinical classification of cardiac deaths. Circulation. 1982;65:457–464. doi: 10.1161/01.cir.65.3.457. [DOI] [PubMed] [Google Scholar]
  • 16.Chugh SS, Socoteanau C, Reinier K, Waltz J, Jui J, Gunson K. A community-based evaluation of sudden death associated with therapeutic levels of methadone. Am J Med. 2008;121:66–71. doi: 10.1016/j.amjmed.2007.10.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Nuckols TK, Anderson L, Popescu L, et al. Opioid prescribing: A systematic review and critical appraisal of guidelines for chronic pain. Ann Intern Med. 2013 Nov; doi: 10.7326/0003-4819-160-1-201401070-00732. 2013 online. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Krebs EE, Becker WC, Zerzan J, Bair MJ, McCoy K, Hui S. Comparative mortality among Department of Veterans Affairs patients prescribed methadone or long-acting morphine for chronic pain. Pain. 2011;152:1789–1795. doi: 10.1016/j.pain.2011.03.023. [DOI] [PubMed] [Google Scholar]
  • 19.Ray WA, Griffin MR. Use of Medicaid data for pharmacoepidemiology. Am J Epidemiol. 1989;129:837–849. doi: 10.1093/oxfordjournals.aje.a115198. [DOI] [PubMed] [Google Scholar]
  • 20.Ray WA. Population-based studies of adverse drug effects. N Engl J Med. 2003;349:1592–1594. doi: 10.1056/NEJMp038145. [DOI] [PubMed] [Google Scholar]
  • 21.Piper JM, Ray WA, Griffin MR, Fought R, Daugherty JR, Mitchel E., Jr Methodological issues in evaluating expanded Medicaid coverage for pregnant women. Am J Epidemiol. 1990;132:561–571. doi: 10.1093/oxfordjournals.aje.a115692. [DOI] [PubMed] [Google Scholar]
  • 22.Dunn KM, Saunders KW, Rutter CM, et al. Opioid prescriptions for chronic pain and overdose. Ann Intern Med. 2010;152:85–92. doi: 10.1059/0003-4819-152-2-201001190-00006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Chung CP, Murray KT, Stein CM, Hall K, Ray WA. A computer case definition for sudden cardiac death. Pharmacoepidemiol Drug Saf. 2010;19:563–572. doi: 10.1002/pds.1888. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Boyer EW. Management of opioid analgesic overdose. N Engl J Med. 2012;367:146–155. doi: 10.1056/NEJMra1202561. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Huikuri HV, Castellanos A, Myerburg RJ. Sudden death due to cardiac arrhythmias. N Engl J Med. 2001;345:1473–1482. doi: 10.1056/NEJMra000650. [DOI] [PubMed] [Google Scholar]
  • 26.Kawai VK, Murray KT, Stein CM, et al. Validation of a computer case definition for sudden cardiac death in opioid users. BMC Research Notes. 2012;5:473. doi: 10.1186/1756-0500-5-473. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Ray WA, Meredith S, Thapa PB, Meador KG, Hall K, Murray KT. Antipsychotics and the risk of sudden cardiac death. Arch Gen Psychiatry. 2001;58:1161–1167. doi: 10.1001/archpsyc.58.12.1161. [DOI] [PubMed] [Google Scholar]
  • 28.Landen MG, Castle S, Nolte KB, et al. Methodological issues in the surveillance of poisoning, illicit drug overdose, and heroin overdose deaths in New Mexico. Am J Epidemiol. 2003;157:273–278. doi: 10.1093/aje/kwf196. [DOI] [PubMed] [Google Scholar]
  • 29.Letsky MC, Zumwalt RE, Seifert SA, Benson BE. Cause of death conundrum with methadone use. A case report. Am J Forensic Med Pathol. 2011;32:193–196. doi: 10.1097/PAF.0b013e3181d3de94. [DOI] [PubMed] [Google Scholar]
  • 30.Arnold SF. Mathematical statistics. Prentice-Hall. 1990:157. [Google Scholar]
  • 31.Ray WA, Murray KT, Kawai V, et al. Propoxyphene and the risk of out-of-hospital death. Pharmacoepidemiol Drug Saf. 2013;22:403–412. doi: 10.1002/pds.3411. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Shadish WR, Steiner PM. A primer on propensity score analysis. Newborn & Infant Nursing Reviews. 2010;10:19–26. [Google Scholar]
  • 33.Austin PC. An introduction to propensity score methods for reducing the effects of confounding on observational studies. Multivariate Behav Res. 2011;46:399–424. doi: 10.1080/00273171.2011.568786. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Arbogast PG, Ray WA. Performance of disease risk scores, propensity scores, and traditional multivariable outcome regression in the presence of multiple confounders. Am J Epi. 2011;174:613–620. doi: 10.1093/aje/kwr143. [DOI] [PubMed] [Google Scholar]
  • 35.Arbogast PG, Kaltenbach L, Ding H, Ray WA. Adjustment of multiple cardiovascular risk factors with a summary risk score. Epidemiol. 2008;19:30–37. doi: 10.1097/EDE.0b013e31815be000. [DOI] [PubMed] [Google Scholar]
  • 36.Arbogast PG, Ray WA. Use of disease risk scores in pharmacoepidemiologic studies. Statistical Meth Med Res. 2009;18:67–80. doi: 10.1177/0962280208092347. [DOI] [PubMed] [Google Scholar]
  • 37.Allison PD. Survival Analysis Using SAS. A Practical Guide. 2nd ed. Cary, NC: SAS Institute; 2010. [Google Scholar]
  • 38.Ray WA. Evaluating medication effects outside of clinical trials: New-user designs. Am J Epidemiol. 2003;158:915–920. doi: 10.1093/aje/kwg231. [DOI] [PubMed] [Google Scholar]
  • 39.Hakkinen M, Launiainen T, Vuori E, Ojanpera I. Comparison of fatal poisonings by prescription opioids. Forensic Sci Int. 2012;222:327–331. doi: 10.1016/j.forsciint.2012.07.011. [DOI] [PubMed] [Google Scholar]

RESOURCES