Real-world integration of the protocol for responding to and assessing patients’ assets, risks, and experiences tool to assess social determinants of health in the electronic medical record at an academic medical center

Abstract

Objective

To describe the real-world deployment of a tool, the Protocol for Responding to and Assessing Patients’ Assets, Risks, and Experiences (PRAPARE), to assess social determinants of health (SDoH) in an electronic medical record (EMR).

Methods

We employed the collection of the PRAPARE tool in the EMR of a large academic health system in the ambulatory clinic and emergency department setting. After integration, we evaluated SDoH prevalence, levels of missingness, and data anomalies to inform ongoing collection. We summarized responses using descriptive statistics and hand-reviewed data text fields and patterns in the data. Data on patients who were administered with the PRAPARE from February to December 2020 were extracted from the EMR. Patients missing ≥ 12 PRAPARE questions were excluded. Social risks were screened using the PRAPARE. Information on demographics, admittance status, and health coverage were extracted from the EMR.

Results

Assessments with N = 6531 were completed (mean age 54 years, female (58.6%), 43.8% Black). Missingness ranged from 0.4% (race) to 20.8% (income). Approximately 6% of patients were homeless; 8% reported housing insecurity; 1.4% reported food needs; 14.6% had healthcare needs; 8.4% needed utility assistance; and 5% lacked transportation related to medical care. Emergency department patients reported significantly higher proportions of suboptimal SDoH.

Conclusions

Integrating the PRAPARE assessment in the EMR provides valuable information on SDoH amenable to intervention, and strategies are needed to increase accurate data collection and to improve the use of data in the clinical encounter.

Background

Utilizing measures of social and environmental determinants of health (SDoH) in the electronic medical record (EMR) to inform population health initiatives has gained momentum in recent years. ¹ The Center for Medicare and Medicaid Services encourages the use of technology in a way that improves patient care outcomes by incentivizing providers, including the addition or utilization of SDoH in the EMR in an effort to address health disparities. ² Although a 2015 Institute of Medicine report recommended incorporating specific measures of social and behavioral domains in the EMR, ³ barriers exist including what standardized measures should be used, how to integrate data capture in the clinical workflow, and how to use determinant data to effectively address health disparities and specific outcomes. ⁴

To this end, the National Association of Community Health Centers developed the Protocol for Responding to and Assessing Patients’ Assets, Risks, and Experiences (PRAPARE) tool to create a standardized patient risk assessment based on evidence-based guidelines.^5,6 The intent is to use the information gathered with the PRAPARE to make informed care decisions and prompt social intervention referrals based on patient complexity and identified needs. Further, integrating social factors with healthcare delivery can facilitate social intervention referrals (e.g. enabling services)^7–9 that improve outcomes and address disparity. ¹⁰ This also relates to one of the concepts of “Learning Health Systems,” where organizations collect, examine, and adapt strategies based on data analytics. However, large healthcare systems (such as academic medical centers) are currently grappling with how to screen and use SDoH data as well as how to seamlessly link high-risk patients to local social services and resources to improve patient and population health outcomes. Moreover, many academic medical centers are moving toward a learning health systems model—where institution-generated data is “fed” back into the system to inform and improve outcomes. Recent work at our institution establishing a Learning Health System service in response to the COVID-19 pandemic highlighted the multiple layers of implementation between healthcare data and conducting research. ¹¹

While several studies have reported on the collection of the PRAPARE,^6,12–15 most have involved collection in community health centers with underserved populations, and none have supplemented their patient-reported data with publicly available area-level SDoH data, which can provide contextual information on patient populations. Further, prior published initiatives have been intentional in nature, often with implementation strategies and metrics to evaluate feasibility with scientific rigor. However, while not ideal, tools such as the PRAPARE are often integrated organically into the clinical workflow at large medical institutions—frequently due to time or resource constraints—which precludes the ability to analyze feasibility with evidence-based measures. These approaches are often spearheaded by a practice champion and inspired by needs being observed in real-time, with experts in evaluation brought in later. This is particularly evident in Learning Health System (LHS) models, where institutions are interested in utilizing real-world data collected in the clinical setting to improve health outcomes.

In January 2020, under the guidance of author HB, the PRAPARE tool was integrated as a template into our health system's EMR system (Cerner). The overall goal was to initiate the collection of SDoH via the PRAPARE at a medical encounter and to refine data collection as integration occurred in real time. Here we describe our experience of this organic, real-world integration of the PRAPARE tool at a large academic medical institution; characterize the SDoH observed in the first year of data collection including the addition of area-level SDoH; and describe lessons learned from preliminary data collection that were used to inform ongoing administration of the PRAPARE tool in our health system.

Methods

Study population

Collection of the PRAPARE was employed in January 2020 on the UAB ambulatory service by the social work division. The ambulatory service includes patients seen at UAB either through a community health clinic or an emergency clinic. Cases are referred to social workers from the ambulatory clinics via the message center to the social work message pool and then allocated to the appropriate social worker assigned by service. Social workers completed a PRAPARE on every patient referred to social work if there was no previous PRAPARE in the EMR at the time of encounter. Approximately 6 months later, collection of the PRAPARE expanded to the Registered Nurse care coordinators in the ambulatory service who managed specific chronic disease types (e.g. diabetes, congestive heart failure, and chronic obstructive pulmonary disease). Recent expansion has included the health system's level one trauma center and the psychiatric emergency services departments. Since the PRAPARE was administered as a part of routine care, this study was deemed exempt by the institutional review board and informed consent was waived/not provided.

PRAPARE tool

The PRAPARE consists of 21 questions across four “domains”: personal characteristics, family and home, money and resources, and social and emotional health (see Appendix 1 for paper assessment tool). The PRAPARE can be given as a paper version to the patient to complete; however, integrating the tool into the EMR and clinical workflow is an important area in development.^9,16,17 For the purpose of this study, we only describe the responses of those who answered ≥ 12 questions on the PRAPARE. The average time to complete the PRAPARE is 9 minutes.

PRAPARE administration

Prior to integrating the PRAPARE in the EMR, HB facilitated the training of 55 social workers, expanding to 10 RN care coordinators in the subsequent phase. Questions on the PRAPARE were reviewed one by one with the staff who also completed an online training module prior to administering the instrument. Scripts for presenting the PRAPARE to patients were created to facilitate the collection around the sensitive nature of the questions. During the initial phase of administration, multiple group meetings were convened to discuss barriers and facilitators of administering the PRAPARE, with adjustments made to administration procedures as needed based on feedback, informing workflow processes and subsequent adjustments.

The team worked closely with the UAB Information Technology (UAB IT) to integrate a dedicated template in the EMR, and the PRAPARE was administered face-to-face with the patient and entered directly into the EMR at the time of collection. During the initial integration period, HB received a monthly data pull, which was examined by the team for data accuracy and template adjustment if needed. HB and staff examined the data monthly to understand the rate of reporting suboptimal SDoH and how to adjust community resource utilization. In addition, HB met regularly with staff to identify challenges in workflow integration, use of data by clinical staff, and adjustments to changing protocols due to the COVID-19 pandemic.

PRAPARE and neighborhood data availability

The PRAPARE is collected and housed via a template in the EMR; thus, providers have access to the data for use in clinical care visits. In addition, data can be requested and extracted for specific research investigations and populations. One question on the PRAPARE queries the patient for an address to facilitate geocoding and linkage of census tract of residence to area-level publicly available data. UAB routinely geocodes patient addresses at the census tract level; therefore, investigators can request census tract for each patient at the time of PRAPARE assessment. Here we linked several area-level indicators to describe the integration sample and to show the utility of the data. We used the Social Vulnerability Index (SVI), ¹⁸ which is a composite index using census data indicators on social factors, that is used to describe the social conditions that may influence human suffering and financial hardship (social vulnerabilities) for disaster planning. Because the SVI is a composite index, we also used the Yost Neighborhood SES (nSES)^19–24 at the census tract level, which utilizes seven components from the census that cover categories of socioeconomic status (SES) including education, income and home values, and employment status. Rurality was determined using the 2010 US Department of Agriculture (USDA) Rural–Urban Commuting Area codes. ²⁵

Evaluation

We informally queried clinical staff (e.g. hospital rounds style) on the use of the PRAPARE tool, including thoughts on workflow implications and the use of SDoH in routine care delivery. We summarized the 9 months of aggregated data obtained via a data pull from UAB IT and sent to the study team. After initial data cleaning, we ran distributions of each PRAPARE question to examine the level of missingness and any unique response patterns in the data, noting any discrepancies. To address missing data, we applied the criteria of answering 12 or more questions on the PRAPARE as useable data for summarization. We also hand reviewed any questions endorsed as “other” and categorized them as appropriate into response categories. This also allowed us to look at common patterns in the “other” category that might inform response/question modification or data collection procedures. We then characterized the study population using descriptive statistics (both overall and by ambulatory admit type) and summarized responses to all PRAPARE questions, keeping separate categories for those missing and marked “choose not to answer.” We discussed the findings among the team to problem-solve ways to improve data collection and/or training of administering the PRAPARE. Because we initiated the collection on 1 January 2020, we had N = 1356 completed before 15 March 2020—roughly the start of the COVID-19 pandemic in the USA. This allowed us to compare the prevalence of SDoH prior to and during the pandemic and to report on modifications to the workflow and administration of the tool.

Results

The flow of assessments is shown in Figure 1. From 1 February to 31 December 2020, N = 10,392 PRAPARE assessments were administered and captured in the EMR. Of those, n = 2705 filled out multiple assessments. Using the first completed PRAPARE assessment for analysis, we excluded an additional n = 1156 assessments that had more than 12 questions missing, leaving a total of n = 6531. The mean age of our descriptive sample was 54.3 years of age, over half were female (58.6%), and approximately 44% were Black (Table 1). Patients who did not have complete PRAPARE data were younger overall and were more likely to be male or seen through the emergency clinic as opposed to community clinics than those with complete data but did not differ by race (Table 1).

Figure 1.

Flow of assessments administered in the implementation phase of the Protocol for Responding to and Assessing Patients’ Assets, Risks, and Experience (PRAPARE) from 1 February to 31 December 2020. Of the N = 10,392 assessments administered, 6531 were summarized after removing multiple assessments (n = 2705) and those with missing data (n = 1156).

Table 1.

Characteristics of the study population.

	PRAPARE a data (n = 6531)	No PRAPARE data (n = 1156)*	p-value b
Age at assessment	54.33 (17.76)	52.16 (19.46)	0.0002
Gender
Male	2693 (41.23)	540 (46.71)	0.0005
Female	3829 (58.63)	612 (52.94)
Missing	9 (0.14)	4 (0.35)
Race, N (%)
American Indian/Alaska Native	4 (0.1)	1 (0.09)	0.624 c
Asian	151 (2.3)	25 (2.2)
Black or African American	2851(43.8)	491 (42.5)
Hispanic or Latino	99 (1.50)	20 (4.7)
White	3260 (50.1)	581 (50.2)
Multiple/other	11 (0.2)	3 (0.2)
Declined/unknown	126 (1.9)	30 (2.6)
Missing	29 (0.4)	5 (0.4)
Admittance type, N (%)
Community medicine clinic	4934 (75.5)	522 (45.2)	<0.0001
Emergency room	1461 (22.4)	557 (48.2)
Other	136 (2.1)	77 (6.6)

*Of the 1156, n = 249 filled out <12 questions and n = 907 were unable to participate.

^a Protocol for Responding to and Assessing Patients’ Assets, Risks, and Experiences (PRAPARE) tool.

^b Chi-square and two-sample t-tests were used for categorical and continuous variables.

^c Fisher's exact test was used to calculate p-value.

In terms of practice patterns and workflow processes, clinic staff reported that the best practice for administering the PRAPARE questions was to incorporate them conversationally into the psychosocial assessment being performed. Staff reported that this felt less like an interview or an invasion on the patient's privacy. Other successful approaches included explaining the survey and allowing the patient to refuse to answer any questions that made them feel uncomfortable. Clinic staff reviewed patient responses and made appropriate referrals if the patient wanted help (e.g. transportation help, linking to food pantries, stress management, and medication consults). Staff reported that the PRAPARE was generally well-received by patients and that most patients answered without hesitation or frustration. However, some patients disliked how personal the questions were and were annoyed that the questions were not directly related to their medical visit. During the peak of the COVID-19 pandemic, the PRAPARE was administered via telephone or telehealth visit but was otherwise collected face-to-face.

The responses to the individual PRAPARE questions are reported in Table 2. In terms of personal characteristics, we found that most of our sample was either Black or White race and that about 95% spoke English. For family and home SDoH, 5.1% indicated not having any housing while about 8% reported housing insecurity. Almost 12% had less than a high school education; over half were either unemployed, not seeking work (46.5%), or disabled (18.8%); and 17.1% did not have insurance. In terms of lacking resources, 1.4% reported food needs, 14.6% reported medicine/healthcare needs, and 8.4% reported needing assistance with utility or phone costs. While most had transportation (80.5%), 5% reported lacking transportation, which kept them from getting medical care, and 7.1% reported lacking transportation to get non-medical necessities. In terms of social and emotional health, over 26% reported high levels of stress, and 5.5% indicated that had social interactions < once per week. Approximately 7% did not feel safe where they currently resided. Missingness ranged from 0.4% (race) to 20.8% (income). Questions that had >5% missing included the following: number living in home (5.4%), education (7.6%), insurance (10.6%), income (20.8%), afraid of partner (7.2%), and incarceration in the last year (5.8%). Many patients endorsed “choose not to answer” on questions; while shown in separate categories here, future analyses will combine missing with choose not to answer. When comparing responses by admit type, we found that in general, those that were admitted via the emergency service were more likely to experience poor SDoH compared to those admitted via community health management across all PRAPARE domains.

Table 2.

Prevalence of social determinants of health collected via the PRAPARE ^a , overall, and by admittance type.

PRAPARE: Personal characteristics	Overall, n = 6531	Community medicine, n = 4934	Emergency room, n = 1461	p-value b
Race, N (%)
American Indian/Alaska Native	5 (0.07)	3 (0.06)	2 (0.14)	0.0309
Asian	128 (1.9)	112 (2.3)	16 (1.1)
Black or African American	2868 (43.9)	2143 (43.4)	662 (45.3)
Hispanic or Latino	98 (1.5)	78 (1.6)	19 (1.3)
White	3258 (49.9)	8 (0.16)	3 (0.2)
Multiple/other	11 (0.1)	2459 (49.8)	732 (50.1)
Declined/unknown	133 (20.3)	110 (2.2)	19 (1.3)
Missing	30 (0.4)	21 (0.4)	8 (0.5)
Migrant work in last 2 years, N (%)
Yes	42 (0.6)	29 (0.6)	13 (0.9)	<0.0001
No	6146 (94.1)	4601 (93.2)	1415 (96.8)
Choose not to answer	92 (1.4)	64 (1.3)	24 (1.6)
Missing	251 (3.8)	240 (4.9)	9 (0.6)
Veteran status, N (%)
Yes	316 (4.8)	251 (5.1)	55 (3.8)	<0.0001
No	5916 (90.6)	4411 (89.4)	1382 (94.6)
Choose not to answer	105 (1.6)	86 (1.7)	16 (1.1)
Missing	194 (3.0)	186 (3.8)	8 (.5)
Spoken language, N (%)
English	6182 (94.6)	4631 (93.9)	1423 (97.4)	<0.0001
Non-English	234 (3.6)	201 (4.0)	30 (2.1)
Choose not to answer	4 (0.06)	4 (0.1)	0 (0.00)
Missing	111 (1.7)	98 (2.0)	8 (.6)
PRAPARE: Family and home
Number living in home, N (%)
1	1680 (25.7)	1112 (22.5)	535 (36.6)	<0.0001
2	2529 (38.7)	1959 (39.7)	514 (35.2)
3–6	1711 (26.2)	1312 (26.6)	359 (24.6)
7+	85 (1.3)	62 (1.3)	19 (1.3)
Choose not to answer	174 (2.6)	156 (3.2)	16 (1.1)
Missing	352 (5.4)	333 (6.7)	18 (1.2)
Housing status, N (%)
No Housing	333 (5.1)	109 (2.2)	216 (14.8)	<0.0001
Have Housing	6052 (92.7)	4710 (95.5)	1218 (83.4)
Choose not to answer	73 (1.1)	47 (.9)	24 (1.6)
Missing	73 (1.1)	68 (1.4)	3 (.2)
Housing insecurity, N (%)
Worried	518 (7.9)	251 (5.1)	256 (17.5)	<0.0001
Not worried	5792 (88.7)	4523 (91.7)	1150 (78.7)
Choose not to answer	108 (1.6)	62 (1.3)	43 (2.9)
Missing	113 (1.7)	98 (2.0)	12 (.8)
PRAPARE: Socioeconomic status
Education, N (%)
<High school	751 (11.5)	439 (8.9)	293 (20.1)	<0.0001
High school	2583 (39.5)	1943 (39.4)	576 (39.4)
>High school	2212 (33.9)	1688 (34.2)	477 (32.7)
Choose not to answer	491 (7.5)	418 (8.5)	68 (4.7)
Missing	494 (7.6)	446 (9.0)	47 (3.2)
Employment, N (%)
Unemployed	562 (8.6)	359 (7.3)	192 (13.1)	<0.0001
Part time	280 (4.3)	190 (3.8)	85 (5.8)
Full time	1034 (15.8)	778 (15.8)	238 (16.3)
Not seeking work (retired/student)	3035 (46.5)	2293 (46.5)	679 (46.5)
Disabled	1227 (18.8)	1018 (20.6)	177 (12.1)
Choose not to answer	224 (3.4)	155 (3.1)	62 (4.2)
Missing	169 (2.6)	141 (2.9)	28 (1.9)
Insurance, N (%)
None/uninsured	1114 (17.1)	663 (13.4)	416 (28.5)	<0.0001
Medicaid	819 (12.5)	580 (11.8)	224 (15.3)
Medicare	1259 (19.3)	1015 (20.6)	221 (15.1)
Commercial/private	2647 (4.5)	2006 (40.7)	592 (40.5)
Missing	692 (10.6)	670 (13.6)	8 (.5)
Family Income
Median family income	$13,000	$15,000	$1052	<0.0001 b
Answered Income question	2353 (36.1)	2006 (40.7)	300 (20.5)	<0.0001
Choose not to answer	2816 (43.1)	2218 (45.0)	524 (35.9)
Missing	1362 (20.8)	710 (14.3)	637 (43.6)
PRAPARE: Resources, N (%)
Report food needs	95 (1.4)	69 (1.4)	24 (1.6)	<0.0001
Report medicine/healthcare needs	955 (14.6)	851 (17.3)	92 (6.3)
Report phone or utility needs	550 (8.4)	389 (7.9)	148 (10.1)
Report childcare needs	15 (0.3)	11 (0.2)	4 (0.3)
Report general lack of money	40 (0.6)	32 (0.6)	5 (0.4)
Report no needs	4876 (74.7)	3582 (72.6)	1188 (81.3)
Transportation, N (%)
No transportation, kept from getting medical care	319 (4.9)	222 (4.5)	87 (5.9)	<0.0001
No transportation, kept from non-medical necessities	464 (7.1)	219 (4.4)	238 (16.3)
No transportation other*	78 (1.2)	65 (1.3)	11 (.7)
Have transportation	5257 (80.5)	4092 (82.9)	1054 (72.1)
Choose not to answer	141 (2.1)	76 (1.5)	62 (4.2)
Missing	272 (4.2)	260 (5.3)	9 (.6)
PRAPARE: Social and emotional health
Stress level rating, N (%)
Not at all	1525 (23.3)	1268 (25.7)	231 (15.8)	<0.0001
A little bit	1563 (23.9)	1323 (26.8)	210 (14.4)
Somewhat	1396 (21.4)	1042 (21.1)	332 (22.7)
Quite a bit	894 (13.7)	640 (13.0)	228 (15.6)
Very Much	845 (12.9)	437 (8.9)	383 (26.2)
Choose not to answer	202 (3.1)	145 (2.9)	52 (3.6)
Missing	106 (1.6)	79 (1.6)	25 (1.7)
Social integration and support, N (%)
1 or 2 times a week	790 (12.1)	512 (10.4)	263 (18.0)	<0.0001
3 to 5 times a week	2099 (32.1)	1691 (34.3)	365 (25.0)
Less than once a week	358 (5.5)	144 (2.9)	204 (14.0)
More than 5 times a week	3005 (46.0)	2384 (48.3)	561 (38.4)
Choose not to answer	194 (3.0)	137 (2.8)	51 (3.5)
Missing	85 (1.3)	66 (1.3)	17 (1.2)
PRAPARE: Personal safety
Afraid of partner in last year, N (%)
No	4781 (73.2)	3650 (74.0)	1024 (70.1)	<0.0001
Yes	120 (1.8)	59 (1.2)	60 (4.1)
Unsure	50 (0.7)	30 (.6)	18 (1.2)
No partner in last year	862 (13.2)	620 (12.6)	227 (15.5)
Choose not to answer	248 (3.8)	131 (2.7)	110 (7.5)
Missing	470 (7.2)	444 (9.0)	22 (1.5)
Feel safe where live, N (%)
No	437 (6.7)	175 (3.6)	251 (17.2)	<0.0001
Yes	5679 (87.0)	4488 (91.0)	1077 (73.7)
Unsure	121 (1.8)	55 (1.1)	64 (4.4)
Choose not to answer	171 (2.6)	111 (2.3)	56 (3.8)
Missing	123 (1.9)	105 (2.1)	13 (.9)
Incarcerated in last year, N (%)
No	5739 (87.9)	4356 (88.3)	1267 (86.7)	<0.0001
Yes	182 (2.8)	85 (1.7)	88 (6.0)
Choose not to answer	230 (3.5)	144 (2.9)	78 (5.3)
Missing	380 (5.8)	349 (7.1)	28 (1.9)
Refugee status, N (%)
No	6040 (92.5)	4499 (91.2)	1411 (96.6)	<0.0001
Yes	27 (0.4)	22 (0.4)	4 (0.3)
Choose not to answer	145 (2.2)	113 (2.3)	30 (2.0)
Missing	319 (4.9)	300 (6.1)	16 (1.1)
Neighborhood level determinants
Social vulnerability index, c N (%)
Low	1141 (17.5)	895 (18.1)	225 (15.4)	<0.0001
Moderate	1526 (23.4)	1218 (24.7)	278 (19.0)
High	2414 (37.0)	1774 (36.0)	592 (40.5)
Missing	1450 (22.2)	1047 (21.2)	366 (25.1)
Rurality, N (%)
Metropolitan	4392 (67.2)	3305 (67.0)	1006 (68.9)	<0.0001
Micropolitan	391 (6.0)	328 (6.6)	50 (3.4)
Small town	206 (3.2)	175 (3.5)	27 (1.9)
Rural	92 (1.4)	79 (1.6)	12 (0.8)
Missing	1450 (22.2)	1047 (21.2)	366 (25.0)
Yost socioeconomic index, c N (%)
Low	2868 (43.9)	2152 (43.6)	662 (45.3)	0.0003
Moderate	1475 (22.6)	1158 (23.5)	284 (19.4)
High	642 (9.8)	504 (10.2)	127 (8.7)
Missing	1546 (23.7)	1120 (22.7)	388 (26.6)

^a Protocol for Responding to and Assessing Patients’ Assets, Risks, and Experience (PRAPARE) tool.

^b Chi-square and Fisher's exact test used for categorical variables and Wilcoxon test used for income variables.

^c Higher values indicate worse neighborhood-level determinants.

When we compared the prevalence of social risks experienced by our patients before and during the pandemic (Supplemental Table), we found that the proportions reporting no housing or housing insecurity increased during the pandemic. We also found that patients seen during the pandemic tended to have lower SES indicators (higher proportion of unemployed, no insurance, less than high school education, and suboptimal nSES) and a trend toward reporting less social support and feeling unsafe where they lived. Interestingly, the proportion of missing in our data after the start of the pandemic improved—in some cases rather markedly, such as insurance, transportation, migrant, and veteran status.

Lastly, Table 3 summarizes the main lessons learned in administering and dealing with the PRAPARE data and strategies to mitigate issues moving forward. For the employment question, n = 793 patients endorsed “other” with multiple descriptions in the free text field that required hand review. Likewise, questions about resources and transportation led to a lot of hand reviews to classify responses. Almost 43% did not respond to the income question, and the insurance question was not asked on the PRAPARE template. It was instead pulled from a discrete EMR field collected routinely in the health system and contained incomplete data (10% missing). The response categories for the stress question were not intuitively worded and did not follow a Likert scale.

Table 3.

Data collection lessons learned and strategies moving forward.

PRAPARE question	Description of lesson learned	Strategies moving forward
Current work situation	• An open text field resulted in n = 793 “other” responses • After hand review found that data collectors would write options that could have fit into the original response categories on the PRAPARE o Retirees n = 183 should have been coded as unemployed, not seeking work o Suggest an addition of a category: disabled/in process filing disability, n = 425 after hand review	• Train data collectors and healthcare personnel to select appropriate response categories (i.e., select “unemployed, not seeking work” for retirees) • Add a response “Disabled/Seeking Disability”
Gone without resources needed in the last year	• N = 364 marked as “other” • While some fell outside the provided categories, many could have been coded in a discrete data field o Child costs, n = 5 o Food, n = 9 o Housing, n = 25 (added as a response category) o Medical costs, n = 200 o Utilities, n = 3 o Transportation, n = 11 • Created category “reported general lack of money,” n = 40 • EMR allowed for multiple responses, but were not in raw data	• Train data collectors and health care personnel to select appropriate response category (i.e. select “Medical costs needs” for those reporting needing help with expensive medications) • Modify data query to allow for multiple responses to be downloaded in the back-end data • Consider adding USDA food insecurity questions to this section
Lack of transportation specific to medical needs or other necessities	• N = 93 marked as “other” o The PRAPARE question is worded—“Has lack of transportation kept you from medical appointments, meetings, work, or from getting things needed for daily living?” with the choices: • Have transportation • No transportation, kept from getting medical care • No transportation, kept from non-medical necessities o Some patients wanted to indicate no transportation in general which leads to a lot of “other” in the data	• Modify the PRAPARE in future applications o Remove transportation from the resource question o Add a question that first asks “Do you lack transportation?” and then administer follow-up questions
Family Income	• 21% missing and 43% choose not to answer • Median income value is questionable due to the amount of missing and errors in recording • Patients may have confused yearly income with monthly income reporting	• Supplement with area-level data that indicates neighborhood SES • Consider adding the SHED questions that assess financial strain per Kusnoor et al. 11 • Clarify that patients report yearly versus monthly income
Stress	• Response categories do not make intuitive sense (not at all, a little bit, somewhat, quite a bit, very much)	• Modify to Likert scale (never, rarely, sometimes, often, always)

PRAPARE: Protocol for Responding to and Assessing Patients’ Assets, Risks, and Experience; EMR: electronic medical record; USDA: US Department of Agriculture; SHED: Survey of Household Economics and Decision-Making.

Discussion

Here we report on lessons learned during the real-world, organic integration of the PRAPARE at a large academic medical institution, summarize strategies to modify current and future data collection efforts, and describe the preliminary prevalence of SDoH in over 6000 patients seen at our institution. Importantly, our analysis of the preliminary data collected during integration revealed ways to improve the quality of the data collected in our healthcare system.

After a hand review of the text fields on several questions, our team found that many answers should have been endorsed in the PRAPARE original responses, which led to strategies to train data collectors and healthcare personnel to improve data quality. On the employment question, we found that over 400 patients were disabled or in the process of filing for disability when we hand reviewed text fields and thus plan to modify the PRAPARE to include this response category. For the resource questions (e.g. gone without childcare, food, housing, utility, medical, or transportation needs), we realized that while the EMR PRAPARE template allowed for multiple responses (e.g. patient had childcare and utility needs) to be endorsed, the data query to pull the data for reporting needed modification so that all responses would be included in the extraction. Transportation was duplicated on the PRAPARE tool in the resource question and queries about lack of transportation for medical and non-medical necessities. Based on response patterns, we found that patients may be confused by the wording of questions. For future applications, we plan to modify the PRAPARE by removing the transportation category from the resource question and pairing it with the transportation question that asks about medical needs and non-medical necessities. We will also consider supplementing the PRAPARE with food insecurity questions from the USDA Short Form of the Food Security Module. ²⁶ Interestingly, we found that the proportion of missingness to questions improved during the COVID-19 pandemic, which we believe was due to the refining of data collection as integration of the PRAPARE progressed rather than a function of the pandemic.

As with other studies,^6,12,13 we saw a high level of missing or inaccurate data for the income question. For future analyses, we will supplement with area-level data (e.g. census tract measures of Neighborhood Socio-economic Environment, the Centers for Disease Control and Prevention's SVI) to estimate the socioeconomic environment where patients reside. However, because area-level data provide information about groups based on geographic similarities that may not relate to the individual patient (e.g. the ecological fallacy), we have considered also adding two questions from the Survey of Household Economics and Decision-Making (SHED) ²⁷ that assesses financial strain. Kusnoor et al. added these to the PRAPARE and observed high response rates. ¹¹

Several studies have described the collection of the PRAPARE in different settings including direct primary care, community clinics, and across several national healthcare centers used to pilot the PRAPARE.^6,12,13 The most prevalent social need risks across these reports were resource-related and included lack of insurance, unmet medicine and healthcare needs, and being at or below 100% of the federal poverty line. We found similarly high rates of uninsured patients and unmet medical needs, while unreliable income data did not allow us to examine poverty levels. Of concern, high-risk SDoH such as lacking current housing and transportation were identified in our population, particularly among those with an emergency visit, as well as concerning rates of self-reported medium–high- to high-stress levels and reporting social interactions (support) less than once per week, again comparable to previous reports.^6,12,13 Our investigation, along with prior evidence, supports the importance of screening for SDoH at the medical encounter, several of which may be amenable to intervention (e.g. social referral and mental health referral).

We were encouraged that patient feedback on PRAPARE was generally positive and that clinic staff saw the tool not as a burden, but rather as a helpful intervention to identify and refer patients. While clinic staff reviewed patient responses and made appropriate referrals for the patient, we currently do not have a process to “close the loop”—where patients are followed to ensure they actually receive services, assess to what degree they utilize those services, and measure how receipt of services influences health outcomes. Recent research has pointed to improved outcomes once social needs have been met, but future work at our institution will seek to standardize the social referral process and closely track outcomes.

We noted marked variability in SDoH needs between the outpatient clinic and the Emergency Department (ED) care settings. Although not surprising given higher rates of ED utilization among underinsured and uninsured patients, the substantially greater needs identified in the ED are notable from a health system standpoint. Data on specific SDoH provide stakeholders with areas to target resources and intervention efforts. In the context of resource limitations and the need for health system leaders to prioritize provision of tailored resources and social referrals to address suboptimal SDoH, the ED as a setting for administering the PRAPARE instrument is valuable. While we did not find marked variation in neighborhood-level SDoH by clinic group, we did note that approximately 30% of our sample live in a suboptimal neighborhood. Understanding the context of patient populations has become increasingly important in the delivery of medical care, tailoring of prevention and intervention efforts, and educating medical professionals. ²⁸ Not surprisingly, we found that the proportions reporting no housing or experiencing housing insecurity increased during the pandemic, while patients seen during the pandemic tended to have lower SES indicators (higher proportion of unemployed, no insurance, less than high school education, and suboptimal nSES) and a trend toward reporting less social support and feeling unsafe where living. These trends highlight the vulnerabilities seen in healthcare systems that are exacerbated by emergency situations—whether it be a pandemic, natural disaster, or manmade catastrophe—and underscore the need to identify and link these patients to resources prior to these events.

Our findings should be viewed in the context of limitations. First, our study population consisted of patients who were administered the PRAPARE during an initial integration period in the health system. This resulted in a convenience sample that is not necessarily representative of the healthcare system as a whole. Because this project was integrated organically into the health system, our ability to provide full participation eligibility, completion rates, and other feasibility measures is limited. As the instrument was administered at a clinical encounter, patients with suboptimal SDoH limiting their ability to attend an ambulatory care visit have a potential to introduce selection bias; however, as data collection expands to a system-wide approach, we will have data that is more representative of our patient population and hospital catchment area as a whole. Moreover, the initial integration included those referred to social work, limiting generalizability of SDoH findings reported. Further, the amount of missing and refusal to answer makes it difficult to ascertain the true prevalence of SDoH since patients may not have answered some questions due to their sensitive nature (e.g. lack of resources) or if a patient was in danger (e.g. domestic violence). This awareness and sensitivity should be considered and incorporated into the training protocols of administering the PRAPARE. We also highlight that our experience reported here was not conducted as a planned feasibility research study. The integration of the PRAPARE was spearheaded by clinical staff to address a need in the health system, which was the need to screen for social risks to facilitate appropriate referrals. Because the integration arose organically, many parameters (e.g. average time to complete survey, reasons for denying survey, and describing how clinicians used the PRAPARE data) were not collected but are areas for study at our institution in the future.

In sum, we found that multiple suboptimal social determinants were reported in over 6000 patients in our local healthcare system, with a significantly higher prevalence spanning all PRAPARE domains among patients in the ED relative to ambulatory community medicine clinic setting. Importantly, preliminary data analysis of the PRAPARE provided information that was used to refine ongoing and future collection of SDoH, contributing to our institution's Learning Health System framework and ensuring quality of the SDoH collected. While integrating the PRAPARE in the EMR provides valuable information on social determinants, strategies to improve the use of data in the clinical encounter as well as accurate and complete reporting of data are needed.