Trends in the diagnosis of diseases of despair in the United States, 2009–2018: a retrospective cohort study

Abstract

Background and objective

Increasing mortality and decreasing life expectancy in the USA are largely attributable to accidental overdose, alcohol-related disease and suicide. These ‘deaths of despair’ often follow years of morbidity, yet little is known about trends in the clinical recognition of ‘diseases of despair’. The objective of this study is to characterise rates of clinically documented diseases of despair over the last decade and identify sociodemographic risk factors.

Design

Retrospective study using a healthcare claims database with 10 years of follow-up.

Setting

Participants resided nationwide but were concentrated in US states disproportionately affected by deaths of despair, including Pennsylvania, West Virginia and Delaware.

Participants

Cohort included 12 144 252 participants, with no restriction by age or gender.

Outcome measures

Diseases of despair were defined as diagnoses related to alcohol misuse, substance misuse and suicide ideation/behaviours. A lookback period was used to identify incident diagnoses. Annual and all-time incidence/prevalence estimates were computed, along with risk for current diagnosis and patterns of comorbidity.

Results

515 830 participants received a disease of despair diagnosis (58.5% male, median 36 years). From 2009 to 2018, the prevalence of alcohol-related, substance-related and suicide-related diagnoses respectively increased by 37%, 94%, and 170%. Ages 55–74 had the largest increase in alcohol/substance-related diagnoses (59% and 172%). Ages <18 had the largest increase in suicide-related diagnoses (287%). Overall, odds for current-year diagnosis were higher among men (adjusted OR (AOR) 1.49, 95% CI 1.47 to 1.51), and among those with Affordable Care Act or Medicare coverage relative to commercial coverage (AOR 1.30, 1.24 to 1.37; AOR 1.51, 1.46 to 1.55).

Conclusions

Increasing clinical rates of disease of despair diagnoses largely mirror broader societal trends in mortality. While the opioid crisis remains a top public health priority, parallel rises in alcohol-related diagnoses and suicidality must be concurrently addressed. Findings suggest opportunities for healthcare systems and providers to deploy targeted prevention to mitigate the progression of morbidities towards mortality.

Strengths and limitations of this study.

• Increasing mortality due to deaths of despair is well documented in extant research. To our knowledge, this study is the first to provide large-scale insights into the clinical recognition of the morbidities that can ultimately culminate in those deaths. This clinical perspective highlights potential opportunities to intervene in the progression of morbidity towards mortality.

• The study uses a large and inclusive sample. As a result, we are able to identify differential patterns in the diagnosis of substance-related, alcohol-related and suicide-related diagnoses across age and gender lines, which can improve targeted prevention efforts.

• The long administrative surveillance period of 10 years allows us to track changes in the identification of diseases of despair over time, and to compare long-term trends between documented morbidity and mortality.

• While trends in deaths of despair appear to vary by race/ethnicity, details on race/ethnicity were not available for our sample.

• We were unable to directly link disease of despair incidence/prevalence to mortality on an individual level; rather, we compare trends in morbidity and mortality in more general terms.

From 2015 to 2017, there was an annual downward trend in life expectancy in the USA, the longest sustained decline since 1915–1918.^{1 2} Relatedly, researchers have observed a longer, more marked increase in the all-cause mortality of middle-aged white non-Hispanic men and women in the USA between 1999 and 2015, with premature deaths largely associated with ‘deaths of despair’, including suicides, accidental poisonings (eg, opiate overdose) and alcohol-related liver disease (eg, cirrhosis).^{3 4}

This troubling observation has coincided with decades of economic decline for less educated and unskilled workers, stagnant or falling real median wages and family incomes,^{5 6} lower marriage rates,⁷ increases in single-parent households⁸ and disengagement from the labour force.⁹ It is theorised that these changes have fostered growing feelings of despair, that is, disillusionment, precariousness and resignation.3 4 10–12 Despair may in turn trigger emotional, cognitive, behavioural and even biological changes,^13–17 increasing the likelihood of diseases that can progress and ultimately culminate in these deaths of despair.

While this pathway may unfold over years or decades, to date, most studies in this domain have primarily focused on the endpoint of mortality. Limited extant research suggests a parallel rise in associated morbidities,^{3 4} but specific estimates of ‘disease of despair’ morbidity (ie, substance-related disorders, alcohol-related disorders, and suicide ideation and attempts) are lacking. Moreover, little is known about trends in the recognition and documentation of these diseases in the clinical setting. This represents an important gap in the literature; indeed, while primary prevention efforts to address the root causes of societal despair are needed, it may be possible to intervene on the pathway from morbidity to mortality in the clinical setting. An understanding of diagnostic patterns and the association between documented morbidity and mortality is necessary to guide these secondary and tertiary prevention efforts. Thus, the objectives of the present study were to use a large administrative database of healthcare claims to (1) characterise the incidence and prevalence of diseases of despair diagnoses over the last decade and (2) identify individual sociodemographic factors and patterns of comorbidity associated with disease of despair diagnosis. Finally, patterns in morbidity and mortality are discussed.

Methods

This retrospective cohort study used claims data extracted from the administrative databases of Highmark, a large US-based health insurance company. Highmark members are concentrated in states that have been disproportionately affected by deaths of despair, including Pennsylvania, West Virginia and Delaware.^{18 19} The database contains clinical information such as services used and diagnoses assigned, and sociodemographic characteristics including gender, age, home address and insurance coverage details. The study cohort included 12 million individuals who were enrolled in a Highmark health insurance plan between 2007 and 2018, and who had a valid age, gender and home address on file. Over 98.5% of enrolled individuals met inclusion criteria and were included in the final analytical cohort.

Study variables

Diseases of despair were defined as diagnoses related to alcohol use, substance use and suicide ideation/behaviours. International Classification of Diseases (ICD) codes were extracted from claims and classified into variables indicating the presence or absence of a diagnosis within each of the three diagnostic categories of interest. Classifications were adapted from the Healthcare Cost and Utilization Project Clinical Classification Software (HCUP-CCS).²⁰ Code mappings were used for HCUP categories alcohol-related disorders (5.11), substance-related disorders (5.12), and suicide and intentional self-inflicted injury (5.13). Diagnoses related to substance use considered outside the focus of the present study were excluded, specifically, codes related to the use of tobacco and cannabis, and certain non-psychoactive substances (ie, ICD-10 codes F550, F551, F552, F554). Additional indicators were created for overall disease of despair status (diagnosis of any type), and co-occurring diseases of despair status (diagnoses in multiple categories). A 2-year lookback period was used to identify incident diagnoses. If no record of the given diagnosis type was found during the lookback period, the diagnosis was recorded as an incident case. Annual and all-time incidence and prevalence estimates were computed.

Additional clinical comorbidities were calculated for the final year of follow-up (2018). Physical health comorbidities were measured using weighted Charlson Comorbidity Index Scores.²¹ Psychiatric comorbidities were defined using additional HCUP-CCS classifications, and included indicators for adjustment/anxiety disorders (5.1 and 5.2), mood disorders (5.8), and schizophrenia and other psychotic disorders (5.10).

Existing research points to particular age groups as having uniquely high risk for deaths of despair overall and by particular subtype. Thus, individuals in the sample were stratified into age groups to allow for comparison between morbidity and mortality trends. Infants less than 1 year old were analysed separately included in order to capture the effects of maternal substance use. The remaining age groups included 1–17 years, 18–34 years, 35–54 years, 55–75 years and 75+ years old.

Insurance product information was recoded into the following categories: Employer sponsored; individual market Affordable Care Act (ACA) plans; Medicare; Other. Product information was captured on an annual basis. If multiple products were found on file during a single year, the product covering the larger number of months was recorded.

To determine whether cohort members were at risk for being diagnosed with a disease of despair at any given point, the months enrolled during each year of follow-up were recorded for all cohort members. Except for newborns, members were considered ‘enrolled’ for a given year if they were covered for at least 10 months out of the year. Newborns were considered enrolled if coverage was identified within the first 90 days of life.

Data analysis

Incidence and prevalence of diseases of despair

Descriptive statistics were computed for all study variables both overall and stratified by diagnostic status (‘any’ vs ‘no’ disease of despair diagnosis recorded over the course of follow-up). Differences were statistically compared using χ² tests for categorical variables and t-tests for continuous variables. Next, overall and gender-specific and age-specific incidence and prevalence rates of disease of despair diagnosis were computed for years 2009 through 2018, allowing for an initial lookback period beginning in 2007. Rates were calculated by dividing the number of newly identified cases by the number of members enrolled and with no previous diagnosis in the past 2 years (incidence), and the number of current year cases by number of members enrolled (prevalence). For the final year of follow-up, descriptive statistics were also computed for weighted Charlson comorbidity scores and additional psychiatric comorbidities.

The partially overlapping samples z-test²² was used to statistically test if and the degree to which rates of diseases of despair changed over the last decade. The partially overlapping test is designed for the statistical comparison of proportions when data include a combination of paired and unpaired samples, as is often the case in dynamic cohorts extracted from clinical databases.

Predicting individual-level risk for diseases of despair

To identify factors associated with individual-level risk for disease of despair diagnosis during the most recent year of follow-up (2018), logistic regression was used to estimate risk for diagnosis as a function of demographic and enrolment characteristics. ORs and their 95% CIs were computed for all estimates. Analyses were performed using R, V.3.5.2²³ within Highmark’s secure computing environment.

Patient and public involvement

Patients or the public were not involved in the design, conduct, reporting or dissemination plans of this research. The study was approved by the Institutional Review Boards of Penn State College of Medicine and the Allegheny Health Network Research Institute (acting IRB for Highmark Health). Informed consent was waived, as no study participants were contacted. Due to privacy laws, data cannot be made publicly available.

Results

Overall, 4.2% of cohort members (n=515 830) were diagnosed with at least one disease of despair at some point during follow-up. Among these, 54.0% were diagnosed with an alcohol-related disorder, 44.2% with a substance-related disorder and 16.3% with suicide ideation/behaviours; multiple types of diseases of despair diagnoses were recorded for 12.9%. Additional cohort description is presented in table 1.

Table 1.

Descriptive statistics for study cohort, overall and stratified by the presence of disease of despair (DoD) diagnosis at any point during follow up, 2007–2018

	Overall	No DoD	DoD	P value
	n=12 144 252 (100.0%)	n=11 628 422 (95.8%)	n=5 15 830 (4.2%)
	N (%)/M (SD)
Type of DoD diagnosis
Alcohol-related	278 309 (2.3%)	–	278 309 (54.0%)
Drug-related	228 227 (1.9%)	–	228 227 (44.2%)
Suicide ideation/behaviour	84 117 (0.7%)	–	84 117 (16.3%)
Multiple diagnosis types	66 585 (0.5%)	–	66 585 (12.9%)
Gender				<0.001
Women	6 124 183 (50.4%)	5 909 902 (50.8%)	214 281 (41.5%)
Men	6 020 069 (49.6%)	5 718 520 (49.2%)	301 549 (58.5%)
Age at enrolment midpoint (years)				<0.001
0–17	2 666 135 (22%)	2 625 797 (22.6%)	40 338 (7.8%)
18–25	1 539 995 (12.7%)	1 456 629 (12.5%)	83 366 (16.2%)
26–35	1 752 493 (14.4%)	1 677 395 (14.4%)	75 098 (14.6%)
36–45	1 638 938 (13.5%)	1 560 822 (13.4%)	78 116 (15.1%)
46–55	1 754 037 (14.4%)	1 659 732 (14.3%)	94 305 (18.3%)
56–65	1 628 179 (13.4%)	1 547 308 (13.3%)	80 871 (15.7%)
66–75	586 303 (4.8%)	552 101 (4.7%)	34 202 (6.6%)
76–85	366 120 (3%)	345 023 (3%)	21 097 (4.1%)
85+	212 052 (1.7%)	203 615 (1.8%)	8437 (1.6%)
No of years enrolled	4.8 (3.4)	4.7 (3.4)	6.6 (3.4)	<0.001
Most recent enrolment				<0.001
2007–2009	1 467 763 (12.1%)	1 440 453 (12.4%)	27 310 (5.3%)
2010–2012	1 796 257 (14.8%)	1 739 269 (15%)	56 988 (11.0%)
2013–2015	2 528 619 (20.8%)	2 421 676 (20.8%)	106 943 (20.7%)
2016–2018	6 351 613 (52.3%)	6 027 024 (51.8%)	324 589 (62.9%)
Medical insurance type				<0.001
Employer sponsored	9 516 477 (78.4%)	9 105 741 (78.3%)	410 736 (79.6%)
ACA	354 355 (2.9%)	340 900 (2.9%)	13 455 (2.6%)
Medicare	752 594 (6.2%)	699 619 (6%)	52 975 (10.3%)
Other	1 520 825 (12.5%)	1 482 161 (12.7%)	38 664 (7.5%)

If there were multiple values on file for state or type of insurance, the value covering longest period was selected. Affordable Care Act (ACA) plans were available 2014–2018.

Age-specific and gender-specific diagnostic prevalence rates from 2009 to 2018 are presented in figure 1. Aggregate incidence and prevalence rates and statistical comparisons between 2009 and 2018 are discussed in the following sections. Age-stratified and gender-stratified statistics are presented in tables 2 and 3.

Figure 1.

Age-specific and gender-specific diagnostic prevalence rates of disease of despair, 2009–2018.

Table 2.

Comparison of age-specific and gender-specific annual incidence of disease of despair diagnoses, 2009–2018

Age cohort	Men				Women
	2009	2018	P value	Ratio	2009	2018	P value	Ratio
	Any disease of despair diagnosis
>1 year	0.25%	0.52%	<0.001	2.05	0.17%	0.36%	0.001	2.18
1–17 years	0.28%	0.38%	<0.001	1.37	0.27%	0.53%	<0.001	1.96
18–34 years	1.44%	1.73%	<0.001	1.21	0.94%	1.42%	<0.001	1.51
35–54 years	1.21%	1.58%	<0.001	1.30	0.72%	1.01%	<0.001	1.40
55–74 years	1.08%	1.61%	<0.001	1.48	0.53%	0.95%	<0.001	1.78
75+ years	0.99%	1.23%	<0.001	1.25	0.69%	0.85%	<0.001	1.23
	Alcohol-related diagnosis
>1 year	0.03%	0.03%	1.000	0.90	0.00%	0.01%	0.951	–
1–17 years	0.09%	0.05%	<0.001	0.60	0.09%	0.04%	<0.001	0.49
18–34 years	0.93%	0.97%	0.103	1.05	0.45%	0.56%	<0.001	1.26
35–54 years	0.90%	1.03%	<0.001	1.15	0.36%	0.44%	<0.001	1.22
55–74 years	0.84%	1.11%	<0.001	1.32	0.26%	0.42%	<0.001	1.60
75+ years	0.63%	0.76%	0.003	1.19	0.24%	0.25%	0.932	1.07
	Substance-related diagnosis
>1 year	0.23%	0.49%	<0.001	2.14	0.17%	0.36%	<0.001	2.14
1–17 years	0.14%	0.08%	<0.001	0.58	0.11%	0.05%	<0.001	0.47
18–34 years	0.57%	0.72%	<0.001	1.26	0.44%	0.58%	<0.001	1.33
35–54 years	0.37%	0.58%	<0.001	1.59	0.34%	0.50%	<0.001	1.50
55–74 years	0.26%	0.51%	<0.001	2.00	0.24%	0.49%	<0.001	2.04
75+ years	0.32%	0.37%	0.117	1.16	0.39%	0.46%	<0.001	1.20
	Suicide-related diagnosis
>1 year	–	–	–	–	–	–	–	–
1–17 years	0.09%	0.28%	<0.001	3.20	0.13%	0.47%	<0.001	3.77
18–34 years	0.14%	0.42%	<0.001	2.95	0.19%	0.52%	<0.001	2.69
35–54 years	0.09%	0.15%	<0.001	1.81	0.13%	0.20%	<0.001	1.52
55–74 years	0.06%	0.13%	<0.001	2.23	0.07%	0.13%	<0.001	1.75
75+ years	0.08%	0.16%	<0.001	2.09	0.08%	0.14%	<0.001	1.83

Ratios represent 2018 rates relative to 2009 incidence rates. Bolded ratios correspond to statistically significant differences in diagnostic incidence.

Table 3.

Comparison of age-specific and gender-specific annual prevalence of disease of despair diagnoses, 2009–2018

Age cohort	Men				Women
	2009	2018	P value	Ratio	2009	2018	P value	Ratio
	Any disease of despair diagnosis
>1 year	0.25%	0.52%	<0.001	2.05	0.17%	0.36%	<0.001	2.18
1–17 years	0.33%	0.45%	<0.001	1.37	0.32%	0.65%	<0.001	2.04
18–34 years	1.93%	2.66%	<0.001	1.38	1.19%	2.05%	<0.001	1.71
35–54 years	1.72%	2.68%	<0.001	1.56	1.03%	1.68%	<0.001	1.63
55–74 years	1.55%	2.72%	<0.001	1.75	0.72%	1.59%	<0.001	2.21
75+ years	1.27%	1.89%	<0.001	1.49	0.80%	1.18%	<0.001	1.46
	Alcohol-related diagnosis
>1 year	0.03%	0.03%	1.000	0.90	0.00%	0.01%	0.950	–
1–17 years	0.10%	0.07%	<0.001	0.67	0.10%	0.05%	<0.001	0.53
18–34 years	1.12%	1.30%	<0.001	1.16	0.52%	0.72%	<0.001	1.37
35–54 years	1.23%	1.54%	<0.001	1.25	0.52%	0.68%	<0.001	1.29
55–74 years	1.23%	1.85%	<0.001	1.51	0.38%	0.69%	<0.001	1.84
75+ years	0.87%	1.24%	<0.001	1.43	0.31%	0.37%	0.140	1.17
	Substance-related diagnosis
>1 year	0.23%	0.49%	<0.001	2.14	0.17%	0.36%	<0.001	2.14
1–17 years	0.17%	0.09%	<0.001	0.57	0.13%	0.06%	<0.001	0.46
18–34 years	0.85%	1.30%	<0.001	1.53	0.59%	0.96%	<0.001	1.62
35–54 years	0.53%	1.19%	<0.001	2.24	0.48%	0.92%	<0.001	1.93
55–74 years	0.33%	0.88%	<0.001	2.70	0.30%	0.83%	<0.001	2.73
75+ years	0.35%	0.51%	<0.001	1.47	0.42%	0.67%	<0.001	1.59
	Suicide-related diagnosis
>1 year	–	–	–	–	–	–	–	–
1–17 years	0.09%	0.31%	<0.001	3.43	0.14%	0.58%	<0.001	4.17
18–34 years	0.15%	0.49%	<0.001	3.20	0.21%	0.64%	<0.001	3.03
35–54 years	0.09%	0.17%	<0.001	1.89	0.14%	0.23%	<0.001	1.60
55–74 years	0.06%	0.15%	<0.001	2.33	0.08%	0.15%	<0.001	1.88
75+ years	0.08%	0.17%	<0.001	2.11	0.09%	0.17%	<0.001	1.94

Ratios represent 2018 rates relative to 2009 prevalence rates. Bolded ratios correspond to statistically significant differences in diagnostic prevalence.

Any disease of despair diagnosis

Between 2009 and 2018, the annual diagnostic incidence of diseases of despair increased by 44%, and the diagnostic prevalence increased by 68%. Significant increases were seen for both men and women across every age group, although the magnitude of increases varied along age and gender lines. Due to the relatively low base rate of diseases of despair, small increases in the absolute rates (ie, percentage point increases) often translated to large increases in the comparison of 2018 rates with 2009 rates (ie, relative rate increases). While percentage point increases tended to be smaller among women compared with men, relative rate increases tended to be larger among women.

Alcohol-related diagnosis

The overall incidence and prevalence of alcohol-related diagnosis increased by 23% and 37%, respectively. Significant increases were seen for men and women ages 18 and above, but significant decreases were seen for those ages 1–17. The most dramatic increases were seen among those ages 55–74; their prevalence increased by 0.5% percentage points (59% relative increase).

Substance-related diagnosis

The overall incidence and prevalence of substance-related diagnosis increased by 48% and 94%, respectively. As with the trends observed for alcohol-related diagnoses, the incidence and prevalence of substance-related diagnoses increased for all groups except for those ages 1–17; for this group, rates significantly decreased. Relative increases were again highest for those ages 55–74 (prevalence 172% higher; 0.5% percentage point increase) and for infants (prevalence 114% higher; 0.25% percentage point increase).

Suicide ideation/behaviours diagnosis

The incidence and prevalence of suicide-related diagnosis increased for all relevant age groups (infants were excluded) by 149% and 170%, respectively. While the absolute rates of suicide-related diagnosis were lower than other types of disease of despair diagnoses, the relative increases were dramatic. Among those ages 1–17, the prevalence increased by 287%. Among those ages 18–34, it increased by 210%. All other age groups saw a relative increase of at least 70%.

Logistic regression results

Results of logistic regression models predicting the odds for 2018 disease of despair diagnoses as a function of individual and enrolment characteristics are presented in table 4.

Table 4.

Risk for disease of despair diagnosis as a function of demographic and enrolment characteristics, 2018

Variable	Any disease of despair	Alcohol-related diagnosis	Substance-related diagnosis	Suicide-related diagnosis
	Adjusted OR (95% CI)
Men (ref=women)	1.49 (1.47 to 1.51)	2.35 (2.30 to 2.41)	1.23 (1.20 to 1.25)	0.72 (0.70 to 0.75)
Age (ref=18–34)
<1 year	0.18 (0.15 to 0.22)	0.02 (0.01 to 0.04)	0.37 (0.31 to 0.44)	–
1–17 years	0.23 (0.22 to 0.24)	0.06 (0.05 to 0.06)	0.07 (0.06 to 0.08)	0.79 (0.75 to 0.83)
35–54 years	0.92 (0.90 to 0.94)	1.09 (1.06 to 1.13)	0.92 (0.90 to 0.95)	0.35 (0.33 to 0.37)
55–74 years	0.83 (0.81 to 0.84)	1.19 (1.15 to 1.22)	0.66 (0.64 to 0.68)	0.21 (0.20 to 0.23)
75+ years	0.44 (0.42 to 0.46)	0.59 (0.55 to 0.63)	0.34 (0.31 to 0.36)	0.15 (0.13 to 0.17)
Insurance (ref=employer sponsored)
ACA	1.3 (1.24 to 1.37)	1.37 (1.28 to 1.48)	1.33 (1.24 to 1.42)	1.05 (0.91 to 1.21)
Medicare	1.51 (1.46 to 1.55)	1.3 (1.25 to 1.36)	1.73 (1.65 to 1.81)	2.19 (1.98 to 2.43)
Other	0.71 (0.67 to 0.74)	0.73 (0.68 to 0.78)	0.61 (0.57 to 0.66)	0.84 (0.77 to 0.92)

Bold indicates statistical significance, p<0.05.

ACA, Affordable Care Act.

Men had higher odds than women for having any type of disease of despair diagnosis, alcohol-related diagnosis and substance-related diagnosis (adjusted odds ratio (AOR) 1.49, 2.35 and 1.23, respectively), but lower odds than women for suicide ideation/behaviours (AOR 0.72). Age was also significantly associated with risk for all four disease of despair outcomes. After adjusting for gender and insurance type, individuals ages 18–35 had the highest risk for any disease of despair, substance-related diagnosis and suicide-related diagnosis. For alcohol-related disorders, those ages 35–74 were at slightly higher risk than those ages 18–34, and much higher risk than any other group. Finally, type of enrolment was also significantly related to risk. Compared with commercial insurance plans, Affordable Care Act (ACA) plans were associated with approximately 1.3 times higher odds for any diagnosis, alcohol-related diagnosis and substance-related diagnosis, but similar risk for suicide-related diagnoses. Medicare plans were associated with 1.5 times higher odds for any disease of despair and each of the three diagnostic subtypes (AOR range was 1.3–2.2).

Clinical comorbidities

Table 5 includes average weighted Charlson comorbidity scores and diagnostic prevalence rates for adjustment/anxiety disorders, mood disorders, and schizophrenia and other psychotic disorders in 2018, stratified by age, gender and the presence of a disease of despair diagnosis during the same year.

Table 5.

Physical and mental health comorbidities by disease of despair diagnosis, age and gender, 2018

	Charlson comorbidity weighted score			Adjustment/anxiety disorders			Mood disorders			Schizophrenia and other psychotic disorders
	No DoD	DoD	P value	No DoD	DoD	P value	No DoD	DoD	P value	No DoD	DoD	P value
	Mean (SD)			N (%)
Men
<1	0.03 (0.21)	0.24 (0.56)	0.001	–	–	–	–	–	–	–	–	–
1–17	0.10 (0.33)	0.20 (0.62)	<0.001	20 980 (6.3%)	949 (63.9%)	<0.001	6538 (2.0%)	1022 (68.8%)	<0.001	137 (0.0%)	136 (9.2%)	<0.001
18–34	0.09 (0.42)	0.24 (0.78)	<0.001	31 405 (8.8%)	4340 (44.5%)	<0.001	17 307 (4.9%)	3980 (40.8%)	<0.001	597 (0.2%)	618 (6.3%)	<0.001
35–54	0.32 (0.93)	0.80 (1.66)	<0.001	49 603 (10.8%)	5178 (41.1%)	<0.001	26 333 (5.8%)	4094 (32.5%)	<0.001	556 (0.1%)	260 (2.1%)	<0.001
55–74	1.1 (1.87)	2.5 (2.94)	<0.001	39 191 (9.3%)	4021 (34.0%)	<0.001	27 685 (6.6%)	3684 (31.2%)	<0.001	1016 (0.2%)	294 (2.5%)	<0.001
75+	2.86 (2.8)	3.00 (4.57)	<0.001	10 126 (10.3%)	563 (31.0%)	<0.001	9204 (9.4 %)	639 (35.2%)	<0.001	672 (0.7%)	89 (4.9%)	<0.001
Women
<1	0.03 (0.21)	0.17 (0.51)	0.044	–	–	–	–	–	–	–	–	–
1–17	0.08 (0.31)	0.21 (0.50)	<0.001	25 099 (7.9%)	1640 (78.7%)	<0.001	9545 (3.0%)	1756 (84.2%)	<0.001	88 (0.0%)	214 (10.3%)	<0.001
18–34	0.13 (0.46)	0.32 (0.77)	<0.001	67 210 (19.3%)	4495 (61.7%)	<0.001	38 526 (11.0%)	4318 (59.3%)	<0.001	381 (0.1%)	407 (5.6%)	<0.001
35–54	0.36 (0.96)	0.92 (1.72)	<0.001	101 708 (21.2%)	5030 (61.4%)	<0.001	64 072 (13.4%)	4475 (54.6%)	<0.001	604 (0.1%)	294 (3.6%)	<0.001
55–74	0.93 (1.69)	2.06 (2.62)	<0.001	83 373 (18.4%)	4015 (54.8%)	<0.001	62 141 (13.7%)	3728 (50.9%)	<0.001	1243 (0.3%)	273 (3.7%)	<0.001
75+	2.28 (2.45)	3.70 (3.0)	<0.001	29 101 (19.7%)	881 (53.4%)	<0.001	23 102 (15.6%)	779 (47.2%)	<0.001	1776 (1.2%)	135 (8.2%)	<0.001

P values for Charlson scores correspond to t-tests; p value for psychiatric comorbidities correspond to χ² tests.

DoD, disease of despair.

Disease of despair diagnosis was associated with significantly higher mean comorbidity scores and significantly higher prevalence of each psychiatric comorbidity for both men and women across every age group. As expected, chronic health conditions were rare among younger cohort members, yet compared to those without disease of despair diagnoses, the average Charlson score among those with disease of despair diagnoses was at least two times higher for every age group up to 74 years. The range of the diagnostic prevalence of adjustment/anxiety disorders was 6.3% to 21.2% among those without disease of despair diagnoses, and 31.0% to 84.2% among those with diagnoses. Similarly, the diagnostic prevalence of mood disorders ranged from 2.0% to 15.6% among those without disease of despair diagnoses, and 31.2% to 78.7% among those with a diagnosis. Finally, while diagnoses of schizophrenia and other psychotic disorders were very rare among those without disease of despair diagnoses, their diagnostic prevalence ranged from 2.1% to 10.3% among those with diagnoses.

Discussion

Nearly 1 in 20 individuals in our sample were clinically diagnosed with a disease of despair between 2007 and 2018. Similar to observed despair-related mortality, diseases of despair have significantly increased as morbidities over the past decade.²⁴ Tracking with the original finding that deaths of despair have disproportionately affected middle-aged men,³ our data showed the largest absolute increases in overall prevalence for men ages 35–74, followed by women ages 55–74 and 18–34. These findings reinforce the notion that while the opioid crisis remains a top public health priority, parallel rises in alcohol-related diagnoses and suicidality must be concurrently addressed.²⁵ Relatedly, the association between diseases of despair and chronic health conditions at all ages along with very high rates of co-occurring psychiatric conditions underscore the interconnectedness of diseases of despair with overall well-being, and the importance of considering reciprocal interrelationships among these conditions in prevention and intervention efforts.

The relative increases in specific diagnoses seen in infants, youth and young adults were also striking. Among infants, the prevalence of substance-related diagnoses doubled over the course of follow-up. This increase was entirely attributable to neonatal abstinence syndrome and corresponds closely with increases in substance-related disorders among women of childbearing age. Neonatal abstinence syndrome could be argued to be the direct result of a disease of despair among mothers and was therefore included. In addition, the prevalence of suicide ideation and behaviours among youth and young adults at least tripled over the course of follow-up. These findings underscore the importance of targeting vulnerable subpopulations in tailored prevention and early intervention services.

A recent study by Gaydosh et al used nationally representative longitudinal data to examine diseases of despair and their age patterns from adolescence through the late 30s from 1994 to 2017.²⁶ This study demonstrated increases as the cohort aged into their late 30s, across racial and ethnic groups as well as geographical locales. In addition to broadening original views beyond impacting whites in rural locales, the generalised increases in despair documented among this younger cohort forewarn further likely increases in mid-life mortality in the coming decades. Such trends are already beginning to be observed at the population level in the USA. Researchers examining declining life expectancy using data from the US Mortality Database and CDC WONDER have recently established that increased death rates among people in midlife are in fact extending beyond the original demographic characterised by Case and Deaton to all racial and ethnic groups, as well as to suburbs and cities (with the largest relative increases occurring in the Ohio Valley and New England).^{3 27} As these and other findings expand the original deaths of despair phenomenon across demographic and geographical boundaries, it strengthens the aforementioned hypothesis that the crisis is systemically linked to material changes in the US political economy that have broadly affected the working class over the last several decades. Thus, while a better understanding of the clinical manifestation of diseases of despair may inform efforts at targeted intervention and mitigation, it must be acknowledged that the ultimate public health goal must be addressing structural root causes of despair.²⁸

The concept of despair remains largely unstudied, with manifestations in cognitive, emotional, behavioural and biological domains as well as in social and political-economic contexts.¹⁷ For example, Glei et al characterised despair by social and psychological dysfunction such as a lack of purpose in life, a sense of worthlessness, little hope or goals for the future, and perceived social rejection by broader society.⁵ Shanahan et al provided a road map for studying the social contexts that can further strengthen or weaken pathways to despair. Ultimately, as influencing the root causes of the crisis (ie, societal structures that perpetuate socioeconomic disadvantage) will be a politically daunting endeavour, there may be shorter-term benefit in improving the healthcare system’s ability to measure and clinically screen for despair, as opposed to resultant diseases or deaths. Better processes for systemically identifying and tracking despair may allow for clinical interventions to help mitigate progression to despair-related mortality.¹⁷

Strengths and limitations

A significant strength of the current study is the investigation of the upstream diagnoses for deaths of despair in a large cohort with representation of some of the most impacted states, including West Virginia and Pennsylvania. Further, the use of a large claims-based dataset provides insight into health insurance type as a consideration in the development of future potential healthcare interventions. Finally, these data provide information on current diagnosis rates in the context of long-term trends. A limitation of the current study is the incomplete nature of social determinants of health data in administrative health records. Despite the significant influence of these factors on health outcomes, few sources provide data that includes both clinical and social and behavioural factors. To address this gap in clinical records, the Institute of Medicine Committee on Recommended Social and Behavioural Domains and Measures for Electronic Health Records describes a concise panel of standard measures in every patient’s electronic health record.²⁹ These measures include items evaluating depression, alcohol use, and social connection or isolation which would help capture diagnoses of despair. Future research might consider combining clinical records with data originating outside of claims and electronic health records, including direct and indirect measures of social determinants.

By definition, diagnoses of diseases of despair and health insurance enrolment imply access to healthcare, although it is clear additional barriers to treatment exist. Insurance type is an important predictor of disease diagnoses, and diseases of despair are no exception. Individuals with Medicare, for example, had 1.5 times higher odds of having a diagnosis of a disease of despair, followed by those with ACA (1.3 times higher odds). Insurance status is frequently used as a surrogate for measures of socioeconomic status, which are essential to describing health inequalities. Unfortunately, this is an imperfect proxy.³⁰ Employer-sponsored insurance suggests the presence of stable household income, yet the nature of coverage and cost sharing responsibilities of patients vary widely across insurance plans. Further, while expanded insurance coverage under the Affordable Care Act has reduced socioeconomic disparities in healthcare access,³¹ significant barriers remain. Ultimately, that an estimated 87 million working adults in the USA are uninsured or underinsured³² remains a major structural challenge for both understanding the true scope of the diseases of despair crisis and mounting clinical programmes to ensure effective treatment.

Directions for future research

Identifying diseases of despair is critical for interrupting their progression towards deaths of despair, but diagnoses do not necessarily guarantee the provision of appropriate and adequate care for acute problems or general well-being. Thus, future research may build on these findings by quantifying treatment following disease identification, and linking this treatment to longer-term morbidity and mortality. Relatedly, while results from the present study suggest strong associations between diseases of despair and both physical and mental health comorbidities, additional research may build on these exploratory findings in multiple ways. First, identifying the clinical correlates of diseases of despair may present opportunities to improve their timely detection through targeted screening. In addition, integrated approaches to treatment that holistically target a range of physical and mental health symptoms may have improved efficacy. It is likely that health systems that work with vulnerable populations (eg, residents of rural and remote regions, low-income adolescents, etc) may need to co-design primary, secondary and tertiary interventions aimed at addressing diseases of despair. Such interventions may require a wider array of community partners than are typically included in conventional health services. Developing and evaluating effective organisational partnerships and best practices in integrated care of patients with diseases of despair would be a valuable contribution.

Future inquiry could benefit from similar ‘big data’ approaches to understand and model patterns of despair in human populations. For instance, it would be useful to identify ‘hot-spots’ of high prevalence/incidence of diseases of despair diagnoses (and deaths of despair) and examine the socioeconomic conditions in these areas over time. As mentioned, research is increasingly—if somewhat inscrutably—linking mortality with prolonged stagnant or declining socioeconomic conditions. These conditions are associated with decades of globalisation, rising automation, and austerity policies (ie, cuts in social spending) that have exacerbated economic precariousness and despair. Notably, researchers have recently established a direct association between automotive assembly plant closures in 112 manufacturing counties located primarily in the US South and Midwest and an 85% surge in opioid overdose mortality rates among working-age adults 5 years later.³³ Others have, for instance, linked rising rates of death of despair with stagnant minimum wage.^34–36

However, others argue that the phenomenon is more attributable to other contextual factors such as a worsening drug environment^37–39; the collapse of social institutions and weakening of traditional social bonds (eg, reduced church attendance,^{40 41} union membership,⁴² etc)⁴³; hospital closures and shortages (particularly in rural areas)⁴⁴; a generalised cultural rise in loneliness, depression, alienation and anxiety⁴⁵; higher relative gun ownership in rural areas⁴⁶; racial resentment and a growing sense of social status loss⁴⁷ among poor whites; and moral decadence (ie, a ‘loss of virtue’ in ethnically white rural communities).^{48 49} Using big data to build predictive models of neighbourhoods or census tracts where people may be at greatest risk would help clarify current debates about past causation and identify zones of resilience, and also offer a future tool of clinical and public health import.

Researchers might also undertake qualitative and/or mixed methods inquiry to examine subjective perceptions of despair in high prevalence communities. There may be particular value in engaging leaders of community and social service organisations (eg, libraries, emergency rooms, methadone treatment centres, suicide prevention organisations, homeless shelters, Women, Infants, and Children (WIC) clinics, legal services organisations, state police, community centres, county health services, places of worship, etc) who interface with high-risk populations, as well as affected persons and family members. It would be especially useful to better understand general awareness of the diseases of despair concept, local beliefs about causation, perceptions of effective policy solutions at the local, state and national levels, and strategies hospitals can use to most effectively intervene. Moreover, as the COVID-19 pandemic contracts domestic and global economies and requires the implementation of social/physical distancing regimes, it will be important for researchers to examine how and in what ways diseases of despair are affected by prolonged isolation, loss of jobs/benefits, diminution of social protections, lack of mental healthcare, drug abuse, increased domestic abuse and other societal consequences of the virus. Given the complexity of despair, combining quantitative and qualitative approaches may be particularly useful.