Prediction of future health care utilization through note-extracted Psychosocial Factors

Abstract

Background.

Persons with multimorbidity (≥2 chronic conditions) face increased risk of poor health outcomes, especially as they age. Psychosocial factors such as social isolation, chronic stress, housing insecurity, and financial insecurity have been shown to exacerbate these outcomes, but are not routinely assessed during the clinical encounter. Our objective was to extract these concepts from chart notes using natural language processing and predict their impact on healthcare utilization for patients with multimorbidity.

Methods.

A cohort study to predict the 1-year likelihood of hospitalizations and ED visits for patients 65+ with multimorbidity with and without psychosocial factors. Psychosocial factors were extracted from narrative notes; all other covariates were extracted from EHR data from a large academic medical center using validated algorithms and concept sets. Logistic regression was performed to predict likelihood of hospitalization and ED visit in the next year.

Results.

In all, 76,479 patients were eligible; the majority were white (89%), 54% were female, with mean age 73. Those with psychosocial factors were older, had higher baseline utilization, and more chronic illnesses. The four psychosocial factors all independently predicted future utilization (ORs=1.27 to 2.77, c-statistic=0.63). Accounting for demographics, specific conditions, and previous utilization, 3 of 4 of the extracted factors remained predictive (ORs= 1.13 to 1.86) for future utilization. Compared to models with no psychosocial factors, they had improved discrimination. Individual predictions were mixed, with social isolation predicting depression and morbidity; stress predicting atherosclerotic cardiovascular disease onset; and housing insecurity predicting substance use disorder morbidity.

Discussion.

Psychosocial factors are known to have adverse health impacts, but are rarely measured; using natural language processing, we extracted factors that identified a higher risk segment of older adults with multimorbidity. Combining these extraction techniques with other measures of social determinants may help catalyze population health efforts to address psychosocial factors to mitigate their health impacts.

Introduction

Over one-fourth of Americans, and three-fourths of those over 65, live with multimorbidity (≥2 chronic conditions).^{1, 2} Their care accounts for two-thirds of healthcare spending in the United States.³ However, chronic illnesses alone are often not the major contributor to poor health outcomes; rather, social and behavioral factors contribute significantly to worsened quality of life, increased disability, exacerbations of illness, and premature mortality.⁴ These psychosocial complexities (e.g., social isolation, chronic stress, financial insecurity) are also social determinants of health and contribute powerfully to racial/ethnic disparities in outcomes among older adults.^5–8 There is growing awareness of the importance of understanding and managing the most impactful constructs – which we term psychosocial vital signs – in conjunction with biomedical problems that patients develop.^{9, 10}

From the biopsychosocial perspective,¹¹ uncovering the interconnections, overlap, and causal pathways between biomedical, psychological, and social processes remain important gaps. As of yet, these represent untapped opportunities to identify best practices that address the specific and unique barriers to effective chronic disease management for older adults with complex care needs; these include those with multiple chronic illnesses¹² Key psychosocial vital signs for this cohort – those related to illness complications and increased disability – include social isolation, chronic stress, financial insecurity, and housing, among others. These factors are related to the onset of and impact from multimorbidity and affect health and well-being over the lifespan.^5–8 For example, there is strong evidence regarding the importance of addressing financial and food insecurity in effective glycemic management^13–17, and reducing rates of avoidable and intensive levels of health care utilization.^{14, 18} For chronic stress, the relationship may be the precipitating factor for increases to morbidity, especially in atherosclerotic disease.^19–25

Despite their relationship to morbidity, structured assessment of these concepts is infrequent in clinical care. Individual social condition codes using ICD10 and validated questionnaires have shown significant promise in assessing social and psychological constructs, but their implementation outside of research settings is still suboptimal. The National Academy of Medicine assessed behavioral and social constructs that were feasible to collect, available, and scientifically acceptable,²⁶ and several groups have tried to implement collection of these constructs in care settings.²⁷ Initial results are encouraging but limited^28–30: entering and reviewing structured data has a significant cost in the time of medical professionals. Without reliable data, overburdened clinical teams cannot know when it is crucial to collect or obtain this information and when to include it in care decisions. Areal or geographic measures of social determinants have value, but lack specificity in individual prediction.³¹ However, we and others find these vital signs embedded in narrative notes. Batra reviewed 82 studies using natural language processing to extract SDOH from text sources, demonstrating a burgeoning literature for extraction, but limited information about application.³² Greenwald et al. demonstrated that one could mine the narrative notes using simple keywords and consistently find a number of social and behavioral issues in inpatients;³³ and Feller reports success in detecting these concepts in HIV patients.³⁴ We previously found key psychosocial vital signs – housing insecurity, financial insecurity, social isolation, and chronic stress - could be extracted with high precision from clinical notes across older adults with multimorbidity.³⁵ The relationship between these extracted variables and onset of and exacerbations of illness, however, is unclear compared to validated instruments.

The goal of this paper was to extend our previous work in extracting these psychosocial concepts to determine their utility in predicting exacerbations of illness for people who already were at risk from their comorbidities – as measured by unplanned hospitalizations and ED visits - with and without other standard predictors; and to test specific hypotheses for how these psychosocial factors may lead to increased exacerbations through pre-defined analyses.

Methods.

Study Design.

We completed a cohort study to understand the benefit of extracting 4 key psychosocial vital signs on prediction of future utilization within a health system. All data were extracted from the Electronic Health Record (EHR) as either structured (from discrete clinical fields) or unstructured (from narrative notes) data.

Population.

Figure 1 demonstrates the inclusion for the cohort; patients were part of the initial population if they were seen during 2016–2017 (N=665,818), were 65 and older (N=212,448), and had 2 or more chronic conditions recorded (N=76,479) at Oregon Health & Science University, a large academic health system.

Inclusion criteria and selection.

Besides age > 65 by 1/1/2017, patients had to be seen in the qualifying visit (baseline) year and the prediction year; the majority of these visits were ambulatory care visits. Patients also had to have 2 or more chronic diseases that are frequently included in operationalizing multimorbidity reflected on their problem list^{36, 37} for initial inclusion. Patients were excluded if they died before the end of their baseline year.

Feature extraction.

Pre-defined features were extracted from the EHR based their inclusion in previous models predicting outcomes. These included: 1) Demographics, including age, race, ethnicity, and biological sex; 2) individual Chronic Conditions, 3) Multimorbidity using Wei et al’s Multimorbidity-Weighted Index (MWI);^{38, 39} 4) psychosocial vital signs, extracted both by ICD10 codes and through Natural Language Processing; and 5) Utilization (primary outcomes of hospitalizations and ED visits). Race and ethnicity were coded together; due to sample size, we first coded all persons selecting Black race as Black, then those selecting Hispanic ethnicity as Hispanic, then those selecting White race as White, then combining all others. Twelve of 76,000 patients selected Black and Hispanic and were coded as Black.

For Chronic Conditions, we selected 35 conditions previously known to be predictive of unplanned utilization and death in various models. We defined these conditions using standard value sets of ICD10 codes extracted from the Value Set Authority Center where possible. The list of value sets for each condition are provided in the Supplement Digital Content (1).

The four selected Psychosocial factors were extracted two ways: first, by using a set of ICD10 codes and structured data questionnaires; and then, using Natural Language Processing using lexical association expansion starting with regular expressions. The full detail is described elsewhere;³⁵ in brief, the authors used an iterative process using ‘seeds’, or words and phrases likely to indicate the feature. For instance, housing insecurity used ‘housing insecurity’, ‘shelter’, ‘no housing’, ‘on the street’ for an initial run. An unsupervised learning approach based on Google’s word2vec (https://code.google.com/p/word2vec/) was used to learn word embeddings and identify other phrases more common in notes with the original phrases. Evaluation of the seeds is done using an information retrieval approach based on Term Frequency-Inverse Document Frequency (TF-IDF), Relevance, and Inverse Patient Frequency. Validation of the seeds was completed until Precision, or Positive Predictive Value (TruePositives / All Retrieved Documents) was > .90.

Multimorbidity measure.

To simplify the analyses and avoid overfitting, we implemented the Wei Multimorbidity Weighted Index measures. The MWI is a person-centered measure that weights chronic conditions based on their impact on the Short Form-36 physical functioning scale, generating negative coefficients. For ease of interpretation, we used the absolute value (positive coefficients) of the index in a set of predictive models which represents both the burden of chronic disease and physical functioning. In the MWI, chronic conditions (all except 4) are also carried forward to help decrease the underestimation of multimorbidity.

Analyses.

Prediction of subsequent (‘outcome’) year hospitalization and ED visits was the primary analytic goal. Descriptive analysis comparing those with and without the psychosocial factors was performed first. Then, planned logistical analyses were performed in the order listed in Table 1; we evaluated the models using likelihood ratios, c-statistics, and goodness of fit for discrimination and calibration.^40–42 We limited features to avoid overfitting; in general, we had more than 100 observations per feature, a rough estimate for adequate sample size.⁴³ The first analyses used no psychosocial vital signs, focusing on demographics, utilization, chronic conditions, and the weight comorbidity score. In the second stage, psychosocial factors were added to the model first, then each set of features (demographics, utilization, multimorbidity scores) were added consecutively. For conditions, a validated index from Wei were used in the final models in lieu of individual diagnoses. Finally, specific, predefined models were used to test individual hypotheses for the value of psychosocial measurements. These included the set in Table 1, focused on the known relationship between chronic stress and atherosclerotic cardiovascular disease (ASCVD) events; financial insecurity, housing insecurity, and onset of or worsening of diabetes; depression and social isolation together leading to worse outcomes; and others.

Table 1.

Pre-planned analyses

Analysis	Rationale	Model
I.0 Prediction of utilization without psychosocial vital signs	Demographics, prior utilization, and multimorbidity predict future utilization.	ED / Hospitalizations predicted by all features except psychosocial vital signs.
I.1 All psychosocial vital signs to predict utilization	Psychosocial factors are known to exacerbate chronic illnesses.	ED / Hospitalizations predicted by all psychosocial vital signs and other features ; compare to I.0
I.2 Diabetes morbidity and financial insecurity, housing insecurity	Patient costs for caring for diabetes have increased, and diabetes requires consistent places to store medication, leading to diabetes exacerbations	Predict ED / hospitalizations with housing insecurity and financial insecurity
I.3 ASCVD and Chronic stress	Chronic stress increases the risk of ASCVD outcomes	Predict ED/hospitalizations in patients with chronic stress and ASCVD
I.4 Depression and Social isolation	Social isolation and mood disorders generate more exacerbations together	Predict ED/hospitalizations in patients with social isolation and depression
I.5 Substance Use and housing insecurity	The combination of substance use and housing insecurity worsens outcomes	Preduct ED/hospitalizations in patietnts with substance use and housing insecurity

^*ASCVD = Atherosclerotic Cardiovascular Disease; ED=Emergency Department

Sensitivity analyses.

We compared the logistic regression models with and without the psychosocial factors to see the relative model improvement (I.0 vs. I.1), and performed forwards and backwards stepwise inclusion to understand the impact of individual factors on the models. We also included patients with missing data in sensitivity models; these were coded as ‘missing’ for categorical variables or 0 for continuous variables.

Results.

In all, 76,479 patients met the initial criteria of 65 or older with 2 or more chronic conditions and one or more visits in 2016 or 2017. Excluding the 2,139 that died during their baseline year, 74,340 patients remained; 21,367 (27.9%) had one or more Psychosocial factors extracted from their chart. Validation of the extraction was presented previously; the average precision (or Positive Predictive Value) of the terms was .91.³⁵ Table 2 presents the baseline characteristics of those with and without psychosocial factors. Patients with extracted psychosocial factors were much more likely to be hospitalized in prior years, more likely to have ED visits, but equally likely to have office visits. This indicates psychosocial vital signs are concurrent predictors of outcomes. They were similar by race (predominantly white) and sex (55% female) but those with psychosocial factors were slightly older; 56% of patients with no psychosocial factors were less than 76 versus 49% of those with psychosocial factors. Patients with psychosocial factors were also more likely to have hospitalizations, ED visits, or die in the outcome year.

Table 2.

Demographics of included patients by psychosocial factor presence

	No PS Factors N (%)	1+ PS Factor N (%)
Total	55,112 (100%)	21,367 (100%)
Age
66–70	17,435 (31.62%)	3,824 (17.90%)
71–75	13,498 (24.49%)	6,775 (31.71%)
76–80	10,041 (18.22%)	4,761 (22.28%)
81–85	6,570 (11.92%)	2,889 (13.52%)
86–90	4,084 (7.41%)	1,698 (7.95%)
91+	3,494 (6.34%)	1,420 (6.65%)
Race/ethnicity
American Indian/Alaska Native	297 (.54%)	108 (.48%)
Asian	1,488 (2.7%)	599 (2.8%)
Black	639 (1.16%)	270 (1.26%)
Hispanic	1,180 (2.14%)	449 (2.1%)
Multiracial	360 (.65%)	151 (.71%)
Other	1,604 (2.91%)	420 (1.97%)
Pacific Islander	106 (.19%)	31 (.15%)
White	49,438 (89.70%)	19,344 (90.53%)
Female Sex	29844 (54.15%)	11,807 (55.26%)
Psychosocial factors
Chronic Stress	N/A 0 (0%)	15,734 (73.64%)
Social isolation	0 (0%)	559 (2.62%)
Housing insecurity	0 (0%)	4,338 (20.30%)
Financial insecurity	0 (0%)	7,117 (33.31%)
Baseline utilization
1+ Hospitalization	4,987 (9.05%)	7281 (34.08%)
1+ ED visit	2,294 (4.2%)	2,909 (14.38%)
1+ Office Visit	51,607 (93.64%)	20121 (94.17%)
Year before baseline utilization
1+ Hospitalization	2,356 (4.27%)	1,616 (7.56%)
1+ ED Visit	1,083 (1.97%)	1,310 (6.13%)
1+ Office Visit	26,617 (48.3%)	12,303 (57.58%)
Most Frequent Diagnoses
Hypertension	14772 (27.3%)	8434 (41.69%)
Cancer	14143 (26.14%)	7021 (34.71%)
Hyperlipidemia	8189 (15.13%)	5078 (25.10%)
Arthritis	6927 (12.80%)	4419 (21.84%)
Dorsopathy	5597 (10.34%)	3880 (19.18%)
Depression	4102 (7.58%)	3268 (16.16%)
Diabetes	6730 (12.44%)	3135 (15.5%)
CAD	4031 (7.45%)	3035 (15%)
Osteoporosis	3454 (6.38%)	2723 (13.46%)
Anxiety	2500 (4.62%)	2413 (11.93%)
Severe Psychiatric illness	1960 (3.62%)	2196 (10.86%)
Multimorbidity-weighted index (Mean ± SD)	7.7±6.8	12.1±9.5
Outcomes in Prediction Year
1+ Hospitalization	2285 (4.15%)	2480 (11.61%)
1+ ED visit	1687 (3.06%)	2052 (9.6%)
Death*	902 (1.6%)	1028 (4.8%)
Hospitalization, ED visit or Death	4320 (7.8%)	4377 (20.48%)

^*Deaths reported to health system only.

Analysis I.0 modeling results without psychosocial variables are demonstrated in the Supplemental Digital Content (2, Tables A.2 and A.3); models with demographics alone had poor performance (c-statistic=0.53) in predicting hospitalization. Adding individual chronic illnesses and previous utilization improved prediction substantially (c-statistic=0.70), butthe MWI score dramatically outperformed individual chronic illnesses (c-statistic=0.76). Similar results were seen for ED visits (demographics c=0.56; individual diagnoses c=0.76; MWI c=0.80).

Models with the psychosocial factors (I.1) results are shown in Table 3a (for hospitalizations) and 3b (for ED visits). The extracted psychosocial constructs alone were all significant predictors of future hospitalizations (PS-a, c-statistic 0.63), with Odds Ratios (ORs) that ranged from 1.38 (Social Isolation, 95% CI 1.07–1.79) to 2.26 (Chronic Stress, 95% CI 2.11–2.42). Adding race, age and the MWI increased performance to c-statistic 0.75. Individual conditions were modestly correlated with the psychosocial constructs (Table A.6) and a model with individual conditions instead of MWI and PS factors had worse performance (c-statistic 0.70). Adding previous utilization further increased the c-statistic (0.78) with previous hospitalizations and recent ED visits both significant. Thus, after accounting for previous hospitalizations and ED visits, knowing whether a patient was labeled as having chronic stress increased the odds of future hospitalization by 1.43. Goodness of fit was poor across all models (Hosmer-Lemeshow p < .01). Performance overall was only marginally improved from the models without psychosocial factors (Tables Appendix .2–3), with 1–2% of the cases more accurately classified.

Table 3.a.

Models to predict Hospitalizations including Psychosocial Factors

Outcome	1+ Hospitalization
Model	I.1.a Psychosocial factors alone	I.1.b PS, demographics, and MWI	I.1.c. PS, demographics, prior utilization, MWI
Likelihood Ratio (df)	1183.4 (4)	1827.7 (17)	3649.9 (21)
C-stat	0.63	0.75	0.77
Hosmer-Lemeshow p	<.0001	<.0001	<.0001
Variables
Psychosocial factors
Homelessness	2.09 (1.89–2.32) *	1.79 (1.61–1.99) *	1.53 (1.37–1.71) *
Social isolation	1.38 (1.07–1.79) *	1.13 (0.86–1.48)	1.02 (0.77–1.36)
Chronic Stress	2.26 (2.11–2.42) *	1.72 (1.6–1.85) *	1.43 (1.33–1.55) *
Financial insecurity	1.64 (1.49–1.79) *	1.54 (1.4–1.69) *	1.36 (1.24–1.5) *
Age (ref 66–70)
71–75		0.82 (0.75–0.89) *	0.84 (0.77–0.92) *
76–80		0.73 (0.66–0.81) *	0.76 (0.69–0.84) *
81–85		0.77 (0.68–0.86) *	0.8 (0.72–0.9) *
86–90		0.7 (0.61–0.8) *	0.71 (0.62–0.82) *
91+		0.89 (0.77–1.03)	0.88 (0.76–1.02)
Race/ethnicity (ref White)
Native American		0.94 (0.6–1.46)	0.87 (0.56–1.36)
Asian		0.91 (0.74–1.11)	0.92 (0.75–1.13)
Black		1.03 (0.78–1.34)	1.01 (0.77–1.32)
Hispanic		1.05 (0.84–1.3)	1.01 (0.81–1.26)
Multiracial		1.1 (0.77–1.58)	1.08 (0.75–1.55)
Pacific Islander		1.13 (0.56–2.28)	1.08 (0.53–2.22)
Female		0.85 (0.8–0.91) *	0.85 (0.8–0.91) *
Multimorbidity weighted index		1.08 (1.07–1.08) *	1.07 (1.07–1.07) *
Hospitalization before baseline			1.12 (1.05–1.19) *
… during baseline			1.51 (1.45–1.57) *
Previous ED visit			1.02 (0.95–1.1)
… during baseline			1.15 (1.1–1.2) *

For ED visit prediction, model performance was slightly improved over hospitalizations (c-statistics=.65-.81). Psychosocial factors alone remained highly predictive (ORs 1.27–2.77); adding race, age, sex and comorbidity scores increased the model performance with moderate impact on PS odds ratios. The exception was social isolation, which no longer significantly predicted outcomes. Older age (86+) and Black and Hispanic race/ethnicity increased odds of ED visits. The MWI remained highly predictive, with each 1 point increase associated with an increased odds ratio between 1.05–1.07. The final models with utilization showed a further increase in prediction, with a 2 year history of ED visits useful in prediction (OR 1.4 for year prior to baseline; OR 2.2 for baseline) but only the baseline years’ hospitalizations (OR 1.06) were predictive.

Individual predictions.

Impact for specific combinations of conditions and psychosocial factors The pre-planned individual prediction models are shown in Table 4 and in the Supplemental Digital Content (2, Tables A.4–A.5). For patients with diabetes at baseline, hospitalizations were slightly better predicted than the overall model with similar odds ratios. ED visits were not better predicted. For patients with ASCVD, chronic stress was a strong predictor of hospitalization and ED visits alone but moderate in adjusted models. Social isolation was a predictor with depression for ED and hospitalization unadjusted models, but it did not reach significance in the adjusted models. Housing insecurity and substance use disorder were strong predictors of ED and hospitalization in unadjusted but not adjusted models; and mental health scores predicted increased, not decreased, office visits.

Table 4.

Pre-specified models with conditions and psychosocial vital signs

Individual	I.2 Diabetes		I.3 ASCVD		I.4 Depression		I.5 Substance Use
	Hospitalization	ED	Hospitalization	ED	Hospitalization	ED	Hospitalization	ED
C-statistic	0.6	0.6	0.63	0.66	0.53	0.56	0.55	0.56
Condition	Diabetes		ASCVD		Depression		Substance Use
Condition result	2.54 (2.31–2.79) *	2.80 (2.53–3.1) *	2.45 (2.29–2.62)	2.85 (2.66–3.06) *	2.59 (2.02–3.33) *	2.49 (1.93–3.22) *	3.1 (2.82–3.41) *	3.18 (2.88–3.52) *
PS Factors	1. Housing, 2. Financial		Chronic Stress		Social isolation		Housing
PS Result 1	2.18 (2.01–2.36) *	1.67 (1.52–1.84) *	1.96 (1.82–2.1) *	2.24 (2.08–2.42) *	1.52 (1.39–1.67) *	2.4 (2.2–2.62) *	1.76 (1.5–2.07) *	2.65 (2.28–3.08) *
PS Result 2	1.52 (1.41–1.64) *	1.73 (1.60–1.99) *
Full: Likelihood Ratio	720.6 (19)	730.3 (19)	674.7 (21)	782.0 (21)	359.2 (21)	542.2 (21)	105.2 (21)	213.6 (21)
C-stat	0.76	0.77	0.71	0.76	0.73	0.74	0.73	0.79
Hosmer-Lemeshow p	<.0001	<.0001	0.0683	<.0001	0.003	0.067	0.09	0.06
Homelessness	1.63 (1.3–2.03) *	1.48 (1.16–1.89) *	1.39 (1.16–1.66) *	1.59 (1.41–1.79) *	1.43 (1.11–1.84) *	1.17 (0.91–1.5)	1.41 (0.92–2.16)	1.2 (0.78–1.85)
Social isolation			1.01 (0.65–1.57)	1.25 (0.93–1.67)	1.27 (0.85–1.9)	1.02 (0.68–1.53)	1.22 (0.5–2.99)	1.12 (0.45–2.79)
Chronic Stress			1.68 (1.48–1.92) *	2.08 (1.92–2.26) *	1.5 (1.24–1.81) *	1.86 (1.56–2.22) *	1.68 (1.18–2.4) *	1.27 (0.89–1.81)
Financial insecurity	1.74 (1.43–2.11) *	1.49 (1.19–1.85) *	1.24 (1.05–1.46) *	1.18 (1.06–1.32) *	1.48 (1.18–1.85) *	1.2 (0.96–1.5)	0.66 (0.42–1.04)	1.05 (0.68–1.62)
Age (ref 66–70)
71–75	0.83 (0.68–1.02)	0.82 (0.66–1.02)	0.96 (0.8–1.15) *	0.76 (0.69–0.84) *	0.88 (0.7–1.11)	0.75 (0.6–0.94) *	1.1 (0.74–1.63)	0.57 (0.37–0.88) *
76–80	0.99 (0.8–1.23)	0.85 (0.67–1.08)	0.9 (0.74–1.09) *	0.78 (0.7–0.87) *	0.74 (0.56–0.98) *	0.9 (0.7–1.15)	0.75 (0.43–1.3)	0.6 (0.35–1.03)
81–85	1.1 (0.86–1.4)	0.93 (0.72–1.22)	0.91 (0.74–1.12)	0.88 (0.78–0.99) *	1.07 (0.79–1.45)	0.96 (0.72–1.29)	0.52 (0.25–1.08)	0.96 (0.54–1.7)
86–90	0.88 (0.63–1.21)	1.13 (0.82–1.56)	0.75 (0.59–0.96) *	1.21 (1.06–1.38) *	1.13 (0.78–1.64)	1.43 (1.03–1.99) *	0.99 (0.43–2.29)	1.61 (0.8–3.24)
91+	1.29 (0.93–1.79)	1.32 (0.94–1.85)	0.88 (0.69–1.13)	1.43 (1.24–1.64) *	1.03 (0.68–1.56)	1.1 (0.75–1.61)	0.81 (0.27–2.4)	1.64 (0.72–3.74)
Race/ethnicity (ref White)
Native American	0.81 (0.36–1.83)	0.37 (0.11–1.23)	0.92 (0.43–1.98)	0.84 (0.51–1.4)	0.77 (0.23–2.59)	0.16 (0.02–1.25)	0.54 (0.07–4.44)	n/a
Asian	0.71 (0.46–1.08)	0.9 (0.59–1.36)	1.27 (0.89–1.81)	1.05 (0.84–1.3)	0.71 (0.37–1.37)	1.07 (0.63–1.82)	n/a	2.56 (0.53–12.32)
Black	0.91 (0.55–1.51)	1.49 (0.94–2.37)	1.25 (0.81–1.92)	1.76 (1.36–2.27) *	1.29 (0.67–2.48)	1.35 (0.72–2.53)	1 (0.4–2.47)	3.08 (1.51–6.31) *
Hispanic	0.98 (0.68–1.4)	1.34 (0.94–1.9)	1.15 (0.77–1.7)	1.33 (1.07–1.65) *	1.05 (0.58–1.89)	1.18 (0.69–2)	0.25 (0.03–1.95)	0.55 (0.12–2.42)
Multiracial	0.82 (0.38–1.77)	1.42 (0.71–2.88)	1.01 (0.51–1.99)	1.08 (0.71–1.62)	0.6 (0.21–1.76)	1.04 (0.43–2.49)	0.83 (0.18–3.74)	0.76 (0.18–3.35)
Pacific Islander	1.22 (0.46–3.26)	1.13 (0.38–3.33)	1.57 (0.58–4.26)	1.26 (0.59–2.69)	2.45 (0.29–20.47)	2.24 (0.27–18.8)	15.38 (1.08–219.8)	0.68 (0.01–95.58)
Female	0.87 (0.75–1.01)	1.01 (0.86–1.18)	0.9 (0.79–1.02)	1.02 (0.95–1.1)	0.86 (0.72–1.04)	0.99 (0.83–1.18)	0.9 (0.64–1.26)	0.93 (0.67–1.29)
hospitalization before baselin	1.23 (1.1–1.36) *	0.88 (0.77–1.01)	1.16 (1.07–1.26) *	1.04 (0.96–1.12)	1.28 (1.13–1.46) *	0.9 (0.77–1.06)	1.16 (0.94–1.43)	1.05 (0.84–1.31)
… during baseline	1.65 (1.53–1.78) *	1.19 (1.1–1.3) *	1.34 (1.26–1.42) *	1.07 (1.02–1.12) *	1.48 (1.33–1.64) *	1.18 (1.06–1.31)	1.55 (1.32–1.82) *	1.15 (0.97–1.37)
Previous ED visit	0.9 (0.77–1.06)	1.46 (1.24–1.72) *	1.08 (0.96–1.21) *	1.57 (1.44–1.71) *	1.01 (0.89–1.14)	1.08 (0.94–1.24)	0.97 (0.83–1.12)	0.98 (0.86–1.11)
… during baseline	1.21 (1.11–1.31) *	2.24 (2.01–2.49) *	1.24 (1.16–1.33) *	2.37 (2.25–2.5) *	1.08 (0.98–1.2)	1.97 (1.76–2.19)	1.11 (1.02–1.21) *	2.08 (1.76–2.46) *
Wei score	1.07 (1.06–1.08) *	1.07 (1.05–1.08) *	1.05 (1.04–1.06) *	1.03 (1.01–1.04) *	1.05 (1.04–1.07) *	1.05 (1.04–1.06) *	1.03 (1.01–1.06) *	1.03 (1.01–1.06) *

Discussion

Psychosocial factors extracted from natural language processing were predictive of unplanned hospitalizations and ED visits, even when accounting for multimorbidity with a validated score, age, sex, race, and previous utilization. This confirms the value of psychosocial factors in older adults with multimorbidity as well as the value of concepts extracted from narrative notes. The net benefit to the models, however, was small compared to those without psychosocial factors, with only 1–2 / 100 additional patients accurately predicted. Social isolation was infrequently recorded in narrative notes and its effect was limited. Individual condition models with predefined populations showed strong prediction for unadjusted models predicting onset of conditions associated with the psychosocial factors (stress – ASCVD; social isolation – depression), but mixed results in focused models adjusted for other factors. For instance, patients with diabetes had better prediction models for hospitalizations from housing insecurity and financial insecurity, but not for depression and social isolation. Thus, adding psychosocial variables for patients with higher multimorbidity appeared to be a more consistently powerful predictor of utilization than attempting to use individual conditions; this lends itself to use in more comprehensive models of care management and coordination.^{44, 45}

Previous work has looked at the benefits of collecting social determinants of health as psychosocial factors. A review showed increasing rates of structured collection of social determinants in EHRs showed promise in use in prediction and quality improvement, but many barriers exist, including lack of agreement on how, when, and why to collect the data.^46–48 When collected, structured factors do predict important outcomes; recent work shows structured documentation of social determinants through ICD-10 codes was predictive of ED visits.

Others have explored the use of automated extraction of terms related to psychosocial factors. Bejan et al. demonstrated an accurate method of housing insecurity and adverse childhood experience identification over time;⁴⁹ Feller et al. successfully extracted psychosocial terms in patients with HIV;⁴⁷ and Newman-Griffis and Fosler-Lussier had high accuracy scores for extracted psychosocial and disability scores.⁵⁰ A recent review identified 82 studies focused on EHR-based extraction of social determinants of health using NLP methodologies, demonstrating that most SDOH constructs can be extracted using similar rule-based approaches.³² Another recent review analyzed the literature related to the integration of social determinants into electronic health records and their impact on risk prediction applications, showing individual level SDOH outperformed areal measures like the Social Vulnerability Index.³¹ This article adds to this literature by providing evidence that psychosocial factors are useful in prediction even in the relatively short term; and providing comparative benchmarks for the relative gain in prediction overall and for specific use-cases. Given the cost of either individual data collection or extraction, focused efforts to collect/extract these factors and use them in models may make more sense than broad screening.

There are several limitations to this study. First, the data is limited to a single health system. Although the data are enhanced in key ways – deaths, ED visits, and hospitalization information is shared and stored from across the local area – the data are likely fragmented and missing key elements for certain patients. This means that the predictions may be partially attributed to volume of data available rather than the unique meaning of each concept. This is particularly important for the psychosocial factors which are not collected in a standard fashion but were extracted from notes; the times and reasons health professionals mention these factors are likely not at random, but tied to the frequency and interactions with the health care system. The diagnosis codes themselves, however, are also used in relation to health system visits, and are overrepresented in frequent visits. The modeling also showed poor goodness of fit; in part, this was due to the fact the two populations – with and without psychosocial factors – are quite different, yielding uneven probability distributions. Focusing on a subset of the population – older adults with multimorbidity – also likely skews the models. This segment, however, is both the fastest growing and most likely to show near term outcomes; the variability in these outcomes make additional factors, like psychosocial variables, particularly valuable to plan potential interventions. Others have used additional socio-economic and geocoded social determinants such as the Social Vulnerability Index⁵¹ and Area Deprivation Index;⁵² however, they may have limited effect when individual variables are available.⁵³

Future work is needed to better integrate these approaches. Determining when and how to screen for /collect structured data for psychosocial factors is still unclear; some reports note that people may be reluctant to ask for help and care teams may see these factors as outside care. Automated extraction could further help decide when individual level collection would be helpful; understanding the sociotechnical aspects of these solutions would be crucial, however, to develop thoughtful interventions. Recent inclusion of these psychosocial vital signs in the health data standards known as the United States Core Data Interoperability definitions answers more of the how, but the burden of data collection remains. More careful work to segment the population most at-risk into potential actionable strategies and interventions is necessary as well, using both areal and individual factors; collecting data that does not yield action could worsen inequity and cause further frustration. Given the low additional benefit of psychosocial variables when a robust dataset is available, adding psychosocial variables should be tailored to the problem of interest. That is, if the psychosocial variables are hypothesized to affect outcomes, the models should be created to address the hypotheses directly. For instance, food insecurity and diabetes may be most impactful for patients on insulin or with severe diabetes. To move to action, better understanding of the causal frameworks for psychosocial factors in the context of multimorbidity is needed. Advancements in modeling are present to infer causality, but have only been applied to these factors in small studies. More rigorous and thorough work is needed.

In all, psychosocial concepts extracted from natural language processing of clinical notes were highly predictive of future utilization and death after rigorous adjustment for patient factors such as multimorbidity, physical functioning, and prior utilization. These approaches may help us understand the vast disparities in health care and enable us to move toward innovations to better address psychosocial needs and improve our systems of health and health care.

Figure 1.

Table 3.b.

Prediction of Emergency Department visits at 1 year post-baseline.

Outcome	1+ ED visits
Model	I.1.a Psychosocial factors alone	I.1.b PS, demographics, and MWI	I.1.c. PS, demographics, prior utilization, MWI
Likelihood Ratio (df)	1252.5 (4)	2437.9 (17)	3817.9 (21)
C-stat	0.65	0.77	0.81
Hosmer-Lemeshow p		0.0036	<.0001
Variables
Psychosocial factors
Homelessness	2.24 (2.01–2.5) *	1.92 (1.72–2.15) *	1.48 (1.31–1.67) *
Social isolation	1.53 (1.18–1.99) *	1.2 (0.91–1.59)	1.06 (0.78–1.44)
Chronic Stress	2.77 (2.57–2.98) *	2.09 (1.93–2.26) *	1.73 (1.59–1.88) *
Financial insecurity	1.27 (1.14–1.4) *	1.19 (1.07–1.32) *	1.12 (1–1.26) *
Age (ref 66–70)
71–75		0.72 (0.65–0.79) *	0.75 (0.68–0.83) *
76–80		0.68 (0.61–0.76) *	0.73 (0.65–0.81) *
81–85		0.75 (0.66–0.85) *	0.78 (0.69–0.89) *
86–90		1.04 (0.91–1.19)	1.07 (0.93–1.23)
91+		1.43 (1.25–1.64) *	1.36 (1.18–1.56) *
Race/ethnicity (ref White)
Native American		0.93 (0.57–1.53)	0.84 (0.5–1.4)
Asian		1.01 (0.81–1.25)	1.04 (0.84–1.29)
Black		1.83 (1.43–2.33) *	1.65 (1.27–2.14) *
Hispanic		1.48 (1.21–1.83) *	1.34 (1.08–1.67) *
Multiracial		1.19 (0.81–1.75)	1.08 (0.72–1.63)
Pacific Islander		1.48 (0.72–3)	1.18 (0.55–2.55)
Female		0.96 (0.89–1.03)	0.98 (0.91–1.05)
Multimorbidity weighted index		1.08 (1.08–1.08) *	1.07 (1.07–1.08) *
Hospitalization before baseline			0.91 (0.84–0.99)
… during baseline			1.06 (1.01–1.11) *
Previous ED visit			1.38 (1.27–1.51) *
… during baseline			2.22 (2.11–2.34) *