Abstract
Purpose
Opioid misuse is a national epidemic, and Ohio is one of the states most impacted by this crisis. Ohio collects county-level counts of opioid associated deaths and treatment admissions. We jointly model these two outcomes and assess the association of each rate with social and structural factors.
Methods
We use a joint spatial rates model of death and treatment counts using a generalized common spatial factor model. In addition to covariate effects, we estimate a spatial factor for each county that characterizes structural factors not accounted for by other covariates in the model that are associated with both outcomes.
Results
We observed an association of health professional shortage area with death rates and the rate of people 18-64 on disability with treatment rates. The proportion of single female households was associated with both outcomes. We estimated the presence of unmeasured risk factors in the southwestern part of the state and unmeasured protective factors in the eastern region.
Conclusions
We described associations of social and structural covariates with the death and treatment rates. We also characterized counties with latent risk that can provide a launching point for future investigations to determine potential sources of that risk.
Keywords: Bayesian hierarchical modeling, disease mapping, multivariate, spatial analysis, substance-related disorders, opioid
Introduction
The United States is currently in the midst of an opioid epidemic. Approximately 4 million persons report nonmedical use of prescription opioids in the past month, and 400,000 persons report heroin use in the United States [1, 2, 3]. Opioid misuse increases the risk of early death, legal problems, and infectious diseases such as hepatitis C and human immunodeficiency virus. Accidental overdose, trauma, and suicide all contribute to premature death in this population [4, 5, 6, 7, 8, 9]. Drug overdose is now the leading cause of injury-related death in the United States [10]. Nationally, the rate of opioid overdose deaths has increased 200% from 2000 to 2014 [11].
Drug overdose rates in Ohio exceed the national average. In 2016, Ohio ranked second for highest overdose death rate [12]. Recently, there has been an unprecedented loss of life in the state due to the influx of illicitly manufactured fentanyl [13]. The number of individuals enrolled in opioid treatment programs in Ohio is also increasing. From 2011 to 2015, the number receiving methadone in a treatment program increased from 4,568 to 6,147 and the number receiving buprenorphine rose from 1,497 to 7,347 [14]. However, national statistics find that approximately 11% of people aged 12 or older who need specialty substance use treatment actually receive it [15].
Two county-level outcomes related to opioid misuse collected by the state of Ohio are opioid associated deaths and treatment admissions. Both outcomes can be considered proxies for measuring the impact of the opioid misuse in a county. Since both deaths and treatment admissions are consequences of the same underlying problem, they are correlated. Thus, rather than exploring a conditional relationship between treatment admissions and deaths, we instead jointly model both as markers of opioid misuse to explore common spatial patterns. Each outcome provides information about the underlying epidemic but from slightly different perspectives. Rather than focus on a single outcome, we have chosen to model them jointly which allows us to leverage associations between the outcomes for a given county to make joint inferences across outcomes[16]. In addition, we can learn about latent characteristics of a county that reflect spatial variation common to both outcomes through a spatial factor model [17].
Through this framework, we can assess the association of social and structural factors on the rates of death and treatment admission [18, 19]. It is important to understand the relationship between social and structural factors and opioid related outcomes to help inform policy and interventions [19] to promote structural change [18]. However, not all social and structural factors are easily measured. An advantage to the spatial factor model approach is that we are able to estimate unmeasured latent risk which is shared by both outcomes and not explained by other covariates. By identifying areas with higher or lower estimated latent risk, we can direct future research towards trying to determine and measure what those factors might be to better explain the epidemic and characterize differences across counties.
Materials and Methods
Data
In an effort to more adequately assess county-level heterogeneity of the opioid epidemic in Ohio, we jointly model opioid associated deaths and treatment admissions for each of Ohio’s 88 counties. We examine aggregate counts from the three most recent available years, 2013-2015. Death counts were obtained from the Ohio Public Health Data Warehouse Ohio Resident Mortality Data. These counts are publicly available from the Ohio Department of Health website (http://publicapps.odh.ohio.gov/EDW/DataCatalog). The death counts include all resident deaths where poisoning from any opioid is mentioned on the death certificate. A resident death is defined as when the decedent resided in Ohio regardless of place of death. Deaths are counted for the county where the decedent resided at the time of death. Poisoning from any opioid is determined by the presence of ICD-10 multiple cause codes T40.0-T40.4 and T40.6 on the death certificate. Observed county opioid associated death rates are shown in Supplemental Figure 1(a).
Treatment admission counts by patient county of residence were obtained from the Ohio Department of Mental Health and Drug Addiction Services through a data use agreement. Treatment admissions included any residential, intensive outpatient, or outpatient treatment for opioid misuse as identified by the diagnostic codes listed in Supplemental Table 1. Counts do not include patients who present to hospitals to receive treatment for overdose or to any other medical facility to receive treatment for complications from opioid misuse. If a patient had multiple admissions, they are only counted once. Counts were provided separately for those under and over age 21; however, we only consider total admissions in this analysis. As per state policy, county-level treatment counts under 10 were masked or censored. Observed county treatment rates are shown in Supplemental Figure 1(b) where bold outlines denote counties with censored counts for under age 21 treatment admissions. In these instances, the over age 21 count is displayed on the map. Due to a data quality issue, Van Wert County was missing treatment counts and appears gray on the map.
Covariates
We acquired covariate information to account for social and structural characteristics of each county. Covariates were selected to reflect themes identified previously in literature [20, 19]. We obtained county population and demographic characteristics from the 2015 5-year estimates of the American Community Survey from the United States Census Bureau. Sociodemographic covariates were standardized prior to modeling by subtracting the mean and dividing by the standard deviation. In addition to Census information, we collected indicators of whether each county was classified as a Health Professional Shortage Area (HPSA) for mental health professionals by the Health Resources and Services Administration. This variable is publicly available from United States government sources. A list and description of all variables considered in this analysis is in Supplemental Table 2.
Statistical Considerations
In our study of Ohio counties, we have 88 bivariate observations of counts of opioid associated deaths and treatment admissions from 2013-2015. We will take a Bayesian approach to conduct a bivariate areal data analysis at the county-level. There are several complexities associated with modeling these data which we address briefly in this section and fully in the Supplement.
The foundation of our model is a generalized common spatial factor model [17]. That is, we assume there is a common spatial factor or latent variable that captures characteristics about a county that are not accounted for in the mean structure and accounts for spatial dependence with the factors of its neighbors. The spatial factor for a county is then shared across the models for each outcome as in a shared latent variable model [21]. This is a sensible assumption when we believe there are common underlying conditions in a location that are associated with multiple outcomes. In our application, we interpret the spatial factor as latent risk for a county related to both rates of death and treatment that is unexplained by the mean structures of the outcome models.
To formulate our model, we use the generalized common spatial factor model [17] for bivariate Poisson outcomes with spatially varying loadings for death [22]. For both outcomes, we adopt the spatial rates parameterization [23, 24] such that we can estimate the relative rates of death and treatment for each county compared to the statewide average. An intrinsic CAR model for areal data is used to describe the spatial structure in the latent factor and loadings [25, 26]. We also include independent latent factors in each outcome model to capture uncorrelated heterogeneity.
As was mentioned previously, treatment admission counts that were less than 10 for either adolescents or adults were suppressed. In our data set, we have four counties with adolescent counts less than 10 but adult counts greater than 10. This presents a situation analogous to interval censoring and can be incorporated into the likelihood. Here we adapt a censored generalized Poisson regression model [27] to the more general case of interval censoring. The treatment count for Van Wert County was imputed within the Bayesian framework 120 of the analysis.
Full model details are presented in the Supplement. We also provide details in the Supplement regarding prior distributions, identifiability, and computational considerations. The MCMC algorithm was implemented in MATLAB version 2015a and run using a single core of a 128GB node on the Wake Forest High Performance Computing Deac Cluster. The algorithm took approximately eight hours to complete, and the code is included in the Supplement.
Results
In this section, we present a summary of sociodemographic characteristics of Ohio counties and the estimates from our joint model. Summary statistics of the sociodemographic characteristics of Ohio counties are presented in Table 1. In addition, there are 55 counties designated as HPSAs, representing 62% of the state’s counties.
Table 1:
Summary of sociodemographic characteristics of Ohio counties
| Variable | Mean | Standard Deviation | 1st Quartile | 3rd Quartile |
|---|---|---|---|---|
| Population | 131,545 | 214,379 | 37,221 | 123,955 |
| Disability Rate 18-64 (%) | 12.97 | 3.59 | 10.57 | 14.72 |
| Unemployment Rate (%) | 7.96 | 2.09 | 6.88 | 9.20 |
| Percent White (%) | 92.18 | 6.90 | 90.50 | 96.53 |
| Median Age (years) | 40.54 | 3.06 | 39.27 | 41.80 |
| Proportion with Bachelors Degree (%) | 18.85 | 8.12 | 13.50 | 22.35 |
| Proportion of Single Female Households (%) | 6.34 | 1.37 | 5.50 | 7.12 |
We assessed the fit of the model using posterior predictive model checks [28, 29] using a χ2 discrepancy function. For treatment, we omitted the five counties with censored or missing data when computing the posterior predictive p-value. We computed posterior predictive p-values of 0.41 for the death rate model and 0.50 for the treatment rate model. Neither value provides strong evidence of lack of fit so we assume that our model is reasonable. Estimates from the model are presented below with additional results included in the Supplement.
Associations with Death Rates
Figure 1(a) maps the posterior mean estimates of the log standardized mortality ratio for each county. We see higher relative death rates estimated across the southwestern portion of the state and rates close to the state average in the northeast and southern regions. We also notice sections of eastern and northwestern Ohio that have lower than average death rates. However, we note that standardized mortality ratios are relative to the state average, which is 57 deaths per 100,000 residents over this three year period. Posterior p-values for the log standardized mortality ratio are in Supplemental Figure 2(a). P-values close to one provide evidence that the parameter is greater than zero while p-values close to zero provide evidence that the parameter is less than zero.
Figure 1:
Posterior Mean Estimates of Model Parameters for Each County
Posterior mean rate ratios for a one unit change and 95% credible intervals for covariates included in the model are shown in Table 2. Units for continuous covariates are standard deviations, as shown in Table 1. Each 1.37% increase in the proportion of single female households is associated with a 25% increased rate of death. Somewhat counter intuitively, being a county in a Health Professional Shortage Area is associated with a 24% decreased rate of death. For this model, we believe this is likely to be an urban/rural proxy reflecting that death rates appear to be higher in urban areas than rural areas rather than indicating a relationship with access to care.
Table 2:
Posterior Mean Rate Ratios for Death Rates and 95% Credible Intervals
| Variable | Rate Ratio | 95% Credible Interval |
|---|---|---|
| Health Professional Shortage Area | 0.76 | (0.60, 0.97) |
| Disability Rate 18-64 | 1.09 | (0.90, 1.29) |
| Unemployment Rate | 1.08 | (0.91, 1.29) |
| Percent White | 1.07 | (0.94, 1.23) |
| Median Age | 1.08 | (0.97, 1.22) |
| Proportion with Bachelors Degree | 1.04 | (0.88, 1.21) |
| Proportion of Single Female Households | 1.25 | (1.07, 1.47) |
Associations with Treatment Rates
In Figure 1(b), we map the posterior mean estimates of the log standardized treatment rate ratio for each county. In general, we observe the highest treatment rates relative to the state average in southern Ohio. This is consistent with the initial state focus on expanding treatment in the region [30]. As with death rates, the ratios reflect comparison to the state average treatment rate, which is 563 admissions per 100,000 residents over this three year period. Posterior p-values for the log standardized treatment rate ratio are in Supplemental Figure 2(b).
In Table 3, we show posterior mean rate ratios for a one unit change and 95% credible intervals for covariates included in the model. The units for continuous covariates are standard deviations, as shown in Table 1. We see an increase of 51% in the treatment rate for every 3.59% increase in the proportion of residents aged 18-64 on disability. We also observe an increase of 18% in the treatment rate for every 1.37% increase in the proportion of single female households and an increase of 14% for every 6.9% increase in the proportion of white residents.
Table 3:
Posterior Mean Rate Ratios for Treatment Rate and 95% Credible Intervals
| Variable | Rate Ratio | 95% Credible Interval |
|---|---|---|
| Health Professional Shortage Area | 0.90 | (0.74, 1.15) |
| Disability Rate 18-64 | 1.51 | (1.27, 1.80) |
| Unemployment Rate | 1.01 | (0.87, 1.17) |
| Percent White | 1.14 | (1.01, 1.33) |
| Median Age | 0.96 | (0.88, 1.05) |
| Proportion with Bachelors Degree | 1.10 | (0.96, 1.27) |
| Proportion of Single Female Households | 1.19 | (1.03, 1.36) |
Spatial Factors
The posterior mean estimates of the spatial factors are shown in Figure 1(c) for each county. The spatial factor captures the joint association between treatment and death rates as well as spatial dependence between neighboring counties after accounting for the outcome-specific mean trends. For identifiability, we have constrained the sign of the spatial factor such that it is positively associated with the treatment rate. Thus, larger values of the factor represent increased risk. We can think of the estimated spatial factor as a way to quantify unmeasured characteristics about a county that are related to death and treatment associated with opioid misuse. That is, the latent factor is an estimate of the intensity of the true underlying unobserved process that is associated with both death and treatment rates. As scale of a latent factor is arbitrary, we can only examine estimates relative to other counties. In general, we see higher values of the spatial factor in southern and southwestern Ohio. This suggests the presence of risk factors in those geographical regions beyond those which we were able to measure and include in the model. We see relatively lower values in the northwestern and eastern parts of the state, which indicates unmeasured protective factors. Posterior p-values for the spatial factors are in Supplemental Figure 2(c).
The posterior mean estimates of the spatial loadings for death are shown in Figure 1 (d). These have been rescaled and centered for ease of interpretation. From a technical perspective, spatial loadings allow the covariance between death and treatment rates within a county to vary across space. However, interpreting the loadings can also provide epidemiological insight. An estimated loading close to 0 indicates that the death and treatment rates similarly influence the factor. A positive value suggests that the death rate has a disproportionate effect on the factor, and a negative value suggests that the treatment rate has a disproportionate effect on the factor. This also serves to highlight areas with a more extreme death rate than we would expect given the treatment rate in that particular county, or vice versa, areas with a more extreme treatment rate than we would expect given the death rate. Posterior p-values for the loadings are in Supplemental Figure 2(d).
Thus if we consider spatial factors and loadings together, we see positive loadings and positive factors in southwestern Ohio. This implies that the death rate is high compared to the treatment rate in that area and that this area has unmeasured risk factors. Likewise, we see negative loadings with positive factors in southern Ohio where the treatment rate is high compared to the death rate and unmeasured risk factors also exist. However, in eastern Ohio, we observe positive loadings and negative factors, which reflect lower rates of death relative to treatment and unmeasured protective factors in this area.
Discussion
In this paper, we jointly modeled opioid associated deaths and treatment admissions for Ohio counties that occurred during 2013-2015. To do so, we built a joint spatial rates model linked by a common spatial factor that used a CAR structure to capture spatial dependence. To overcome issues associated with censoring due to small counts of treatment admissions, we implemented an interval censored Poisson model to incorporate as much information as possible. Using Markov chain Monte Carlo, we estimated posterior distributions for the parameters of interest.
We identified several social and structural covariates that are associated with death and treatment rates. We found an association between death rates and counties with Health Professional Shortage Area status, which is in many ways indicative of the urban/rural divide. We found associations of the proportion of single female households with both death and treatment rates. We hypothesize that, as single female household is an indicator of higher risk of poverty, the stress associated with economic distress increase ones chance of drug use and overdose while also increasing likelihood of qualifying for federal and state assistance, thus improving access to treatment [31]. We also found associations of the treatment rate with the rate of those 18-64 years old on disability and the proportion of white residents. These findings likely reflect the initial response of increasing treatment availability to areas particularly impacted by prescription opioid misuse [30], which tended to be more rural and have higher rates of chronic health issues [20]. In addition to measurable covariates, we were able to estimate the contribution of unmeasured characteristics to the death and treatment rates. We can visualize the spatial factor on a map to help to identify areas where future research can be directed to identify additional factors to better explain heterogeneity in rates across space. The estimated loadings also point us towards counties where future public health interventions should be targeted to try to mitigate higher than expected death rates. We were able to illuminate all of these findings through joint modeling and use of a spatial factor model.
Our analysis relied on state surveillance data on opioid associated deaths and treatment admissions. Death counts are derived from information recorded on the death certificate which is generally considered imperfect [32]. We have assumed for this analysis that the observed counts are correct and do not account for any misclassification related to information on or omitted from the death certificate. We are also limited by the aggregate and areal nature of the analysis. We were only able to obtain treatment counts that were aggregated across time and thus could not examine any trends over time. The entire analysis is areal and thus conclusions are limited to the county-level and cannot be directly applied to the person-level so as to avoid the ecological fallacy [33].
Conclusion
In this paper, we have shown associations between county-level characteristics and rates of death and treatment due to opioid misuse. We have also identified unmeasured spatial characteristics that are associated with both outcomes and could be used as a launching point for future investigations to find additional characteristics that might help to better explain the epidemic. In addition, we were able to highlight areas where the relative death or treatment rates were higher or lower than expected given the other rate. Taken together, these findings form another piece in the puzzle as we try to further understand the ongoing opioid epidemic in Ohio and effectively appropriate resources to high risk areas.
Supplementary Material
Acknowledgements
Funding: Research reported in this publication was supported by the National Institute On Drug Abuse of the National Institutes of Health under Award Number R21DA045236. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
References
- [1].Center for Behavioral Health Statistics and Quality, Behavioral health trends in the United States: results from the 2014 national survey on drug use and health, Internet, http://www.samhsa.gov/data/sites/default/files/NSDUH-FRR1-2014/NSDUH-FRR1-2014.pdf (July 2015).
- [2].Brady KT, McCauley JL, Back SE, Prescription opioid misuse, abuse, and treatment in the United States: An update., The American journal of psychiatry 173 (1) (2016) 18–26. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [3].Dart RC, Surratt HL, Cicero TJ, Parrino MW, Severtson SG, Bucher-Bartelson B, Green JL, Trends in opioid analgesic abuse and mortality in the United States., The New England journal of medicine 372 (3) (2015) 241–8. [DOI] [PubMed] [Google Scholar]
- [4].Hser YI, Saxon AJ, Huang D, Hasson A, Thomas C, Hillhouse M, Jacobs P, Teruya C, McLaughlin P, Wiest K, Cohen A, Ling W, Treatment retention among patients randomized to buprenorphine/naloxone compared to methadone in a multi-site trial., Addiction 109 (1) (2014) 79–87. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [5].Degenhardt L, Larney S, Kimber J, Farrell M, Hall W, Excess mortality among opioid-using patients treated with oral naltrexone in Australia., Drug and alcohol review 34 (1) (2015) 90–6. [DOI] [PubMed] [Google Scholar]
- [6].Evans E, Li L, Min J, Huang D, Urada D, Liu L, Hser YI, Nosyk B, Mortality among individuals accessing pharmacological treatment for opioid dependence in California, 2006-10., Addiction 110 (6) (2015) 996–1005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [7].Teesson M, Marel C, Darke S, Ross J, Slade T, Burns L, Lynskey M, Memedovic S, White J, Mills KL, Long-term mortality, remission, criminality and psychiatric comorbidity of heroin dependence: 11-year findings from the Australian Treatment Outcome Study., Addiction 110 (6) (2015) 986–93. [DOI] [PubMed] [Google Scholar]
- [8].Centers for Disease Control and Prevention (CDC), HIV infection and HIV-associated behaviors among injecting drug users - 20 cities, United States, 2009., MMWR. Morbidity and mortality weekly report 61 (8) (2012) 133–8. [PubMed] [Google Scholar]
- [9].Degenhardt L, Larney S, Kimber J, Gisev N, Farrell M, Dobbins T, Weatherburn DJ, Gibson A, Mattick R, Butler T, Burns L, The impact of opioid substitution therapy on mortality post-release from prison: Retrospective data linkage study, Addiction 109 (8) (2014) 1306–1317. [DOI] [PubMed] [Google Scholar]
- [10].Chen L, Hedegaard H, Warner M, Quickstat: Rates of death from drug poisoning involving opioid analgesics-United States, 1999-2013, MMWR 64 (1) (2015) 32. [PubMed] [Google Scholar]
- [11].Rudd RA, Aleshire N, Zibbell JE, Gladden RM, Increases in drug and opioid overdose deaths–United States, 2000-2014., MMWR. Morbidity and mortality weekly report 64 (50-51) (2016) 50–51. [DOI] [PubMed] [Google Scholar]
- [12].Hedegaard H, Warner M, Minino A, Drug overdose deaths in the united states, 1999-2016, NCHS Data Brief 294 URL http://wonder.cdc.gov [PubMed]
- [13].Daniulaityte R, Juhascik MP, Strayer KE, Sizemore IE, Harshbarger KE, Antonides HM, Carlson RR, Overdose deaths related to fentanyl and its analogs - Ohio, January-February 2017., MMWR. Morbidity and mortality weekly report 66 (34) (2017) 904–908. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [14].SAMHSA, Behavioral health barometer: Ohio, volume 4: Indicators as measured through the 2015 national survey on drug use and health, the national survey of substance abuse treatment services, and the uniform reporting system, HHS Publication No. SMA-17-Baro-16-States-OH, http://www.samhsa.gov/ (2017). [PubMed]
- [15].SAMHSA, Key substance use and mental health indicators in the united states: Results from the 2016 national survey on drug use and health, HHS Publication No. SMA 17-5044, NSDUH Series H-52, http://www.samhsa.gov/ (2017).
- [16].Martinez-Beneito MA, Botella-Rocamora P, Banerjee S, Towards a multidimensional approach to Bayesian disease mapping, Bayesian Analysis 12 (1) (2017) 239–259. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [17].Wall MM, Wang F, Generalized common spatial factor model, Biostatistics 4 (4) (2003) 569–582. [DOI] [PubMed] [Google Scholar]
- [18].Rhodes T, Singer M, Bourgois P, Friedman SR, Strathdee SA, The social structural production of hiv risk among injecting drug users., Social science & medicine 61 (5) (2005) 1026–44. [DOI] [PubMed] [Google Scholar]
- [19].Perlman DC, Jordan AE, The syndemic of opioid misuse, overdose, hcv, and hiv: Structural-level causes and interventions., Current HIV/AIDS reports 15 (2) (2018) 96–112. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [20].Palombi LC, St Hill CA, Lipsky MS, Swanoski MT, Lutfiyya MN, A scoping review of opioid misuse in the rural united states, Annals of Epidemiology. [DOI] [PubMed] [Google Scholar]
- [21].Dunson DB, Dynamic latent trait models for multidimensional longitudinal data, Journal of the American Statistical Association 98 (463) (2003) 555–563. [Google Scholar]
- [22].Neeley E, Christensen W, Sain S, A bayesian spatial factor analysis approach for combining climate model ensembles, Environmetrics 25 (7) (2014) 483–497. doi: 10.1002/env.2277. [DOI] [Google Scholar]
- [23].Cressie NAC, Wikle CK, Statistics for spatio-temporal data, Wiley, Hoboken, N.J., 2011. [Google Scholar]
- [24].Cressie N, Perrin O, Thomas-Agnan C, Likelihood-based estimation for Gaussian mrfs, Statistical Methodology 2 (1) (2005) 1–16. [Google Scholar]
- [25].Banerjee S, Carlin BP, Gelfand AE, Hierarchical modeling and analysis for spatial data, Chapman & Hall/CRC, Boca Raton, Fla, 2004. [Google Scholar]
- [26].Besag J, Spatial interaction and the statistical analysis of lattice systems, Journal of the Royal Statistical Society. Series B (Methodological) 36 (2) (1974) 192–236. [Google Scholar]
- [27].Famoye F, Wang W, Censored generalized poisson regression model, Computational Statistics and Data Analysis 46 (3) (2004) 547–560. [Google Scholar]
- [28].Gelman A, Meng X-L, Stern H, Posterior predictive assessment of model fitness via realized discrepancies, Statistica Sinica 6 (4) (1996) 733–760. [Google Scholar]
- [29].Stern HS, Cressie N, Posterior predictive model checks for disease mapping models, Statistics in Medicine 19 (17-18) (2000) 2377–2397. [DOI] [PubMed] [Google Scholar]
- [30].Governor’s Cabinet Opiate Action Team, Attacking Ohio’s opiate epidemic, Online; accessed 6-September-2017, http://mha.ohio.gov/ (2012).
- [31].Tucker J, Lowell C, National snapshot: Poverty among women & families, 2015, National Women’s Law Center, https://nwlc.org/wp-content/uploads/2016/09/Poverty-Snapshot-Factsheet-2016.pdf (2016). [Google Scholar]
- [32].Slavova S, O’Brien DB, Creppage K, Dao D, Fondario A, Haile E, Hume B, Largo TW, Nguyen C, Sabel JC, Wright D, Members of the Council of State and Territorial Epidemiologists Overdose Subcommittee, Drug overdose deaths: Let’s get specific., Public health reports 130 (4). [DOI] [PMC free article] [PubMed] [Google Scholar]
- [33].Pianntadosi S, Byar DP, Green SB, The ecological fallacy, American Journal of Epidemiology 127 (5) (1988) 893–904. doi: 10.1093/oxfordjournals.aje.a114892. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.