Skip to main content
PMC home page
Scientific Reports logoLink to Scientific Reports
. 2024 Apr 16;14:8815. doi: 10.1038/s41598-024-57410-6

Measuring social determinants of health in the All of Us Research Program

Samantha Tesfaye 1,2, Robert M Cronin 3, Maria Lopez-Class 4, Qingxia Chen 5, Christopher S Foster 4, Callie A Gu 6, Andrew Guide 5, Robert A Hiatt 7, Angelica S Johnson 8, Christine L M Joseph 9, Parinda Khatri 10, Sokny Lim 11, Tamara R Litwin 1, Fatima A Munoz 12, Andrea H Ramirez 11, Heather Sansbury 13,16, David G Schlundt 14, Emma N Viera 15, Elif Dede-Yildirim 11, Cheryl R Clark 6,✉
PMCID: PMC11021514  PMID: 38627404

Abstract

To accelerate medical breakthroughs, the All of Us Research Program aims to collect data from over one million participants. This report outlines processes used to construct the All of Us Social Determinants of Health (SDOH) survey and presents the psychometric characteristics of SDOH survey measures in All of Us. A consensus process was used to select SDOH measures, prioritizing concepts validated in diverse populations and other national cohort surveys. Survey item non-response was calculated, and Cronbach’s alpha was used to analyze psychometric properties of scales. Multivariable logistic regression models were used to examine associations between demographic categories and item non-response. Twenty-nine percent (N = 117,783) of eligible All of Us participants submitted SDOH survey data for these analyses. Most scales had less than 5% incalculable scores due to item non-response. Patterns of item non-response were seen by racial identity, educational attainment, income level, survey language, and age. Internal consistency reliability was greater than 0.80 for almost all scales and most demographic groups. The SDOH survey demonstrated good to excellent reliability across several measures and within multiple populations underrepresented in biomedical research. Bias due to survey non-response and item non-response will be monitored and addressed as the survey is fielded more completely.

Subject terms: Human behaviour, Risk factors, Epidemiology

Introduction

There is growing scientific consensus that social and environmental factors are significant contributors to health status, including the onset or progression of disease, recovery or response to treatment, and inequities in distributions of illnesses among populations1,2. Investigating the roles of social factors in health is an essential part of precision medicine research.

The All of Us Research Program (All of Us) aims to collect data from over one million participants to advance precision medicine and improve population health3. A central goal of All of Us is to ensure program participants represent the diversity of the US. The program seeks to build a diverse cohort that emphasized recruitment of populations underrepresented in biomedical research (UBR) along the lines of racial and ethnic identity, age, sex, gender identity, sexual orientation, disability status, access to care, income, educational attainment, and geographic location3,4. All of Us supports research that integrates information on genomics and other biologic data, data on lifestyle or behaviors, and contextual factors including the social determinants of health (SDOH) to facilitate precision medicine research. Data from All of Us participants are captured in multiple ways, including health surveys to obtain participant provided information, electronic health record data, physical measurements, and biospecimens. Details of the program’s recruitment efforts and data types have been reported previously3.

To collect data on social factors, All of Us launched a Task Force of subject matter experts within the All of Us Research Program to develop a survey on SDOH. SDOH concepts were selected to provide psychometrically rigorous data with strong use cases relevant to precision medicine. This technical report outlines the approach and process used to construct the SDOH survey. Additionally, this report presents psychometric characteristics of the measures included in the final SDOH survey, based on the most recently released data from 117,783 All of Us participants who submitted the survey by June 30, 2022.

Methods

Survey development process

In 2017, the SDOH Task Force launched for the purpose of developing a survey to gather self-reported data from All of Us participants that captures information on dimensions of SDOH using scientifically valid and reliable scales, while minimizing burden on participants to complete surveys.

A six-phase process was developed by the Task Force to select concepts that could be measured with validity and reliability via self-report in a large, diverse, and multilingual cohort. This process is summarized in Supplementary Table 1. In brief, the Task Force employed a consensus based process to define SDOH and conceptual frameworks to (1) establish the inclusion and exclusion criteria, and priorities to guide construct selection; (2) evaluate relevant scientific literature to identify measures with strong psychometric properties and use cases in precision medicine; and (3) examine SDOH measures used in other biobanks and large epidemiologic studies to find opportunities to align measures with other resources, chiefly the UK Biobank, the Million Veteran Program, and the NIH PhenX Toolkit surveys related to SDOH5–7. Additionally, the Task Force coordinated internally to (4) avoid duplication of other All of Us surveys already deployed or in development (e.g., mental health, environmental/occupational exposures, and a survey offered to all participants as they join the program called “The Basics”) that assess concepts that overlap with or may be considered SDOH (e.g., income, education, employment, health literacy, home ownership, and risk of homelessness)8. The Task Force used conceptual frameworks to improve the user experience and eliminated lengthy scales that added to participant burden. Final measurement selections were completed with recommendations from scientific subject matter experts and All of Us participant partners. Importantly, inclusion and exclusion criteria for surveys prioritized measures of perceptions (including cognitions, beliefs, attitudes) that could only be obtained from the perspective of participants, and not otherwise collected through geocoding, electronic health records, or other means. Priority was given to selecting constructs and measures that are likely to represent core drivers and mechanisms connected to health inequities9,10.

The final survey included concepts and measures that had been previously validated and that had documentation on psychometric performance in large cohort studies. However, in rare instances, measures or items were included where extant literature is emerging when domain areas were considered essential for measurement, namely housing instability and housing quality11. Priority was given to measures that were scientifically validated in multiple languages and cultural contexts. In general, measures were used in the way they were validated with their original response sets. A notable exception was made for the questions on spirituality and religious service attendance, where a “non-religious” response option was added in keeping with strong recommendations from All of Us participants to acknowledge differing views on religion.

Conceptual frameworks and definitions

The SDOH Task Force chose the World Health Organization Conceptual Framework for Action on the Social Determinants of Health as an organizing scientific framework to identify constructs that could be operationalized to establish connections among SDOHs, their origins in structural social, economic, political, historical and cultural factors, measures of social identity and social stratification (income, education, race, ethnicity, sexual orientation and gender identity, disability) and relation to psychological, material resources/health-related social needs, and social connections that influence health or health care utilization1. Additionally, the Task Force sought to align with the Healthy People 2020 framework as described by the Centers for Disease Control and Prevention (CDC) Office of Disease Prevention and Health Promotion to identify five broad domains to refine the measurement approach: social and community context, economic stability, education, neighborhood and built environment, health and health care12. Of these domains, the Task Force identified educational attainment and health care access as domains that were, in part, already covered in other All of Us surveys.

To improve the user experience and ability to communicate well with large, diverse audiences about the relation of social factors to health, the Task Force was guided by the work of the Robert Wood Johnson Foundation Commission to Build a Healthier America, which developed language and concepts to communicate about social factors across political groups13.

Finally, the Task Force solicited guidance and recommendations from All of Us Participant Ambassadors on concepts and specific measures resonant with, or that might be less relevant to, lived experience of participants14. The Task Force also reviewed measures with scientific subject matter experts who were the developers of measurement tools under consideration, as well as subject matter experts from institutes and centers of the National Institutes of Health, who recommended additional concepts and measures for consideration.

Using these three frameworks and recommendations, the Task Force adopted the following as the guiding definition for the survey:

“Social determinants of health are the conditions and context in which people are born, live, learn, play, work, and worship across the lifespan that influence quality of life.”

Cognitive and online pilot testing

Candidate measures that were considered for inclusion in the survey underwent cognitive interviewing and pilot testing prior to finalizing final instruments and items.

Prior to launching the SDOH survey, a pilot study was conducted from January 2020—February 2020 and February 2021—April 2021. The methods of the pilot study included cognitive interviews and online testing, similar to the methods described in the development of the baseline surveys for All of Us15. The pilot study included qualitative and quantitative assessment of the SDOH survey in English and Spanish versions, the two languages that are currently used in All of Us. The pilot study recruited individuals aged 18 and older with a focus on ensuring the sample included the perspectives of UBR populations, particularly those underrepresented due to racial and ethnic identity, income, and preferred language. The Pilot Research Core’s Expression of Interest Registry, Latino Center of the Midlands in Omaha, Nebraska, and Cint (https://www.cint.com/), an online survey audience platform, served as the recruitment methods for reaching key populations to test the survey.

Survey modifications after testing

The cognitive interviews provided feedback on the content and the processes for fielding the survey, which was incorporated in the final SDOH survey (Supplementary Table 2). Content modifications that were implemented included eliminating some concepts from the draft survey. Process recommendations for fielding the survey that were implemented included providing materials so that participants who indicated social needs via survey could be made aware of relevant resources for the issues they reported. Online pilot testing of the survey demonstrated completion times that were in the range of 10–15 min. Pilot completion times were similar among different demographic and language groups (Spanish and English).

Survey approval and release

The final SDOH survey was approved by the All of Us Institutional Review Board (IRB) and launched on November 1, 2021. The SDOH concepts approved in the final survey are listed in Table 1. The SDOH survey is optional and is available in English and Spanish language versions to all participants who have completed the first three All of Us surveys (The Basics, Overall Health, and Lifestyle). Participants select the language in which they take program surveys at the time of program enrollment.

Table 1.

Constructs and measures in the All of Us SDOH survey.

Construct Measure Referencesa
Social and community context
 Social cohesion among neighbors Social cohesion neighborhood scale (4-item) 16
 Social support RAND MOS social support survey (8-item) 17
 Loneliness UCLA loneliness scale (8-item) 18
 Perceived discrimination Everyday discrimination scale (9-item), plus attribution (1-item) 19,20
 Perceived stress Cohen’s perceived stress scale (10-item) 21,22
 Daily spiritual experiences Fetzer multidimensional measure of religiousness/spirituality (6-item) 23
 Religious service attendance Religious service attendance nurses’ health study (1-item) 24
 English proficiency California health interview survey language spoken at home (1-item) 25
Economic stability
 Food insecurity Children’s HealthWatch Hunger vital sign (2-item) 26
 Housing instability indicator Health begins upstream risk screening (1-item) 27
 Housing quality Accountable health communities (1-item) 28
Neighborhood and built environment
 Neighborhood physical disorder Ross–Mirowsky perceived neighborhood disorder scale (6-item physical disorder subscale) 29
 Neighborhood social disorder Ross–Mirowsky perceived neighborhood disorder scale (7-item social disorder subscale) 29
 Neighborhood walkability Physical activity neighborhood environment scale (PANES) (5 core items) 30,31
 Neighborhood crime Physical activity neighborhood environment scale (PANES) crime (2-item subscale) 30,31
Neighborhood residential density Physical activity neighborhood environment scale (PANES) neighborhood housing type (1-item indicator) 30,31
Health and healthcare
 Perceived discrimination in medical care settings Everyday discrimination scale in medical settings (7-item) 32
Open in a new tab

aReferences provide original measure description and validation information or scoring recommendations.

Accessing data from the SDOH survey

The SDOH survey is publicly available online33. The data can be found in the All of Us Researcher Workbench at https://www.researchallofus.org/data-tools/workbench/. The SDOH survey questions, responses with answer concept ids, number, and percentages of participants who selected each response, along with bar charts showing the number of participants who chose each answer by the sex assigned at birth and age when the survey was taken, are publicly available via the All of Us Data Browser at https://databrowser.researchallofus.org/.

Construction of scales

We constructed scales and scored items in alignment with validated usage in the literature. Two exceptions include the presentation of the Religious Service Attendance item and Daily Spiritual Experiences scale, which incorporated All of Us Participant Ambassador feedback by adding response set options for participants to indicate “I am not religious,” or to indicate “a higher power,” as an alternative conceptualization of spiritual experience. Due to a transcription error for the Religious Service Attendance measure, an incorrect response set was displayed for 11,795 participants; instructions to identify these observations are described in the Supplementary Methods. A detailed scoring recommendation for all measures is included in the Supplementary Methods.

Statistical analyses

Characteristics of All of Us participants who responded to the SDOH survey, and those who were eligible but did not submit responses to the SDOH survey, were summarized in means and standard deviations (SD) (continuous variables) or counts and percentages (categorical variables). Among those who responded to the SDOH survey, distributions of the constructed scales were reported in mean, SD, range, median, and interquartile range (IQR). Item non-response was examined by demographic category to understand potential bias in missing data. Item non-response was defined as insufficient items completed by a participant to calculate a scale. In most cases, a scale could not be calculated with more than one or two items not completed; item non-response definitions for each scale are provided in the Supplementary Methods. For each scale, Cronbach's alpha coefficients were used to examine internal consistency overall and by demographic category.

Multivariable logistic regression models were used to examine if demographic categories were associated with item non-response. The predictors in the model included racial identity, Hispanic/Latino/Spanish ethnicity, sex assigned at birth, gender identity, sexual orientation, educational attainment, annual household income, disability status, and current age. Age was modeled with a restricted cubic spline function with three knots. Each model is displayed visually as a forest plot to compare the odds ratios and corresponding confidence intervals (CIs). For each plot, the odds of an individual of a particular demographic category (racial identity, sexual orientation, etc.) not responding to the survey scale of interest are compared to the odds of non-response for an indicated reference population within that variable; this yields an odds ratio of item non-response comparing the two groups. Since age was treated non-linearly, two odds ratios were calculated for age: people aged 25 and 75 were compared to a reference group of age 50 for illustration purposes.

The ‘pandas’, and ‘numpy’ Python packages were used for data cleaning, and the ‘pingouin’ package was used to calculate Cronbach’s alpha for the SDOH scales. The R (version 4.1.1) kernel within the Jupyter environment was used for the remaining analysis. In particular, the 95% CIs for proportion differences were constructed using prop.test function in the ‘stats’ (version 4.2.0) package, regression models were performed using the ‘rms’ (version 6.6) package, and forest plots were generated by the ‘metafor’ (version 4.0) packages.

The current analyses were conducted as part of a demonstration project designed to describe All of Us cohort data in preparation for releasing data to the All of Us Researcher Workbench. The data described in this report are from the All of Us Researcher Workbench version 7 released to the Researcher Workbench in April 2023. The work described here was proposed by Consortium members and confirmed as meeting criteria for non-human subjects research by the All of Us IRB. Results reported are in compliance with the All of Us Data and Statistics Dissemination Policy disallowing disclosure of group counts under 20.

Results

SDOH survey questions

The constructs and measures in the final fielded SDOH survey are listed in Table 1.

Description of survey participants

A total of 397,732 participants were eligible to complete the SDOH survey, of which 332,986 (83.7%) were UBR. As of June 30, 2022, a total of 117,783 (29.6%) participants provided any SDOH survey data, of which 92,300 (78.4%) were UBR.

There were differences in the characteristics of those who answered any portion of the survey (survey respondents) and those who did not answer any portion of the survey (non-respondents, Table 2). Survey respondents were predominantly White as compared to non-respondents (74.6% vs. 44.8%). More respondents had a college or advanced degree than non-respondents (62.5% vs. 36.0%). Compared to survey respondents, non-respondents had lower incomes ($50,000 or less per year, 27.8% vs. 45.0%).

Table 2.

Demographic characteristics of eligible All of Us Research Program participants.

SDOH survey respondentsa (n = 117,783) SDOH survey non-respondentsa (n = 279,949) Difference between respondents and non-respondents (95% CI)
Age, years, Mean (Standard deviation) 59.3 (16.5) 53.1 (17.0) 6.2 (6.1, 6.3)
Characteristic N (%) N (%)
 RBR Overall 25,483 (21.6%) 39,263 (14.0%) 7.6% (7.3%, 7.9%)
 UBR Overall 92,300 (78.4%) 240,686 (86.0%) − 7.6% (− 7.9%, − 7.3%)
Racial Identityb
 White 87,886 (74.6%) 125,452 (44.8%) 26.6% (26.3%, 26.9%)
 Black, African or African American 8984 (7.6%) 65,958 (23.6%) − 16.0% (− 16.2%, − 15.8%)
 Hispanic/Latino/Spanish 7980 (6.8%) 54,776 (19.6%) − 12.8% (− 13.0%, − 12.6%)
 Asian 3117 (2.6%) 9,982 (3.6%) − 1.0% (− 1.1%, − 0.9%)
 Native Hawaiian or Pacific Islander 57 (0%) 346 (0.1%) − 0.1% (− 0.1%, − 0.1%)
 Middle Eastern or North African 467 (0.4%) 1760 (0.6%) − 0.2% (− 0.3%, − 0.2%)
 Multi-Racial 4221 (3.6%) 11,332 (4.0%) − 0.4% (− 0.5%, − 0.3%)
 None of these describe me 1039 (0.9%) 3148 (1.15%) − 0.3% (− 0.3%, − 0.2%)
 Prefer not to answer or skip 4032 (3.4%) 7195 (2.6%) 0.8% (0.7%, 0.9%)
Sex assigned at birth
 Female 74,451 (63.2%) 165,071 (59.0%) 4.2% (3.9%, 4.5%)
 Male 40,354 (34.3%) 109,494 (39.1%) − 4.8% (− 5.1%, − 4.5%)
 Intersex, none of these describe me, prefer not to answer, or skip 2978 (2.5%) 5384 (1.9%) 0.6% (0.5%, 0.7%)
Gender identity
 Woman 73,499 (62.4%) 164,032 (58.6%) 3.8% (3.5%, 4.1%)
 Man 40,013 (34%) 108,950 (38.9%) − 4.9% (− 5.2%, − 4.6%)
 Non-Binary 522 (0.4%) 613 (0.2%) 0.2% (0.2%, 0.2%)
 Transgender 171 (0.1%) 365 (0.1%) 0.0% (0.0%, 0.0%)
 None of these describe me, prefer not to answer, or skip 3578 (3%) 5989 (2.1%) 0.9% (0.8%, 1.0%)
Sexual orientation
 Straight 101,913 (86.5%) 244,515 (87.3%) − 0.8% (− 1.0%, − 0.6%)
 Lesbian 1699 (1.4%) 3213 (1.1%) 0.4% (0.3%, 0.5%)
 Gay 3061 (2.6%) 5990 (2.1%) 0.5% (0.4%, 0.6%)
 Bisexual 4650 (3.9%) 9771 (3.5%) 0.4% (0.3%, 0.5%)
 None of these describe me 2343 (2%) 5948 (2.1%) − 0.1% (− 0.2%, 0.0%)
 Multiple 364 (0.3%) 338 (0.1%) 0.2% (0.2%, 0.2%)
 Prefer not to answer or skip 3753 (3.2%) 10,174 (3.6%) − 0.4% (− 0.5%, − 0.3%)
Educational attainment
 College graduate or advanced degree 73,606 (62.5%) 100,892 (36.0%) 26.5% (26.2%, 26.8%)
 College (Years One to Three) 27,446 (23.3%) 72,400 (25.9%) − 2.6% (− 2.9%, − 2.3%)
 Grade Twelve or GED 10,694 (9.1%) 63,959 (22.8%) − 13.7% (− 13.9%, − 13.5%)
 Less than high school degree or equivalent 2585 (2.2%) 33,136 (11.8%) − 9.6% (− 9.7%, − 9.5%)
 Prefer not to answer or skip 3452 (2.9%) 9562 (3.4%) − 0.5% (− 0.6%, − 0.4%)
Annual household income
 > 150 k 21,212 (18%) 23,240 (8.3%) 9.7% (9.5%, 9.9%)
 100–150 k 18,655 (15.8%) 21,248 (7.6%) 8.2% (8.0%, 8.4%)
 50–100 k 30,963 (26.3%) 43,118 (15.4%) 10.9% (10.6%, 11.2%)
 < 50 k 32,700 (27.8%) 125,917 (45.0%) − 17.2% (− 17.5%, − 16.9%)
 Prefer not to answer or skip 14,253 (12.1%) 66,426 (23.7%) − 11.6% (− 11.8%, − 11.4%)
Disability
 No disabilities identified 51,964 (44.1%) 59,275 (21.2%) 20.9% (20.6%, 21.2%)
 Living with disabilities 15,659 (13.3%) 25,598 (9.1%) 4.2% (4.0%, 4.4%)
 Not assessedc 50,160 (42.6%) 195,076 (69.7%) − 27.1% (− 27.4%, − 26.8%)
Survey language
 English 115,196 (97.8%) 258,827 (92.5%) 5.3% (5.2%, 5.4%)
 Spanish 2587 (2.2%) 21,122 (7.5%) − 5.3% (− 5.4%, − 5.2%)
Open in a new tab

RBR represented in biomedical research; UBR underrepresented in biomedical research.

aRespondents or non-respondents, respectively, are defined as those with or without any responses to the SDOH survey as of June 30, 2022 in the All of Us Research Program Version 7 data release. All of Us participants who completed core baseline surveys were eligible to take the SDOH survey.

bFull racial and ethnic identity data (e.g., Japanese, Cuban) not provided due to space limitation.

cParticipants who responded to the core Basics survey before October 22, 2019 were not surveyed for demographic information on disability.

Internal consistency reliability of scales

Internal consistency measured by Cronbach’s alpha was over 0.8 for almost all scales (Table 3). The multi-dimensional Physical Activity and Neighborhood Environment (PANES) Walking and Bicycling scale had the lowest Cronbach’s alpha at 0.78. Internal consistency did not vary substantially by participant characteristics for most scales (Supplementary Table 4). However, for participants who identify as Native Hawaiian and Pacific Islander (N = 57), the PANES Walking and Bicycling scale had a much lower Cronbach’s alpha (0.58) compared to that of other groups where alpha coefficients for the Walking and Bicycling scale ranged from 0.70 to 0.79. Estimates of internal consistency on other scales for Native Hawaiian Pacific Islanders ranged from 0.78 to 0.95.

Table 3.

Score distributions, item non-response and Cronbach’s alpha of All of Us SDOH survey measures (N = 117,783).

Construct N items Score range Mean (SD) Median (IQR) Item non-response N (%) Cronbach's alpha
Loneliness 8 0–100 31.0 (21.6) 29.2 (12.5–45.8) 3042 (2.6%) 0.87
Social support (total) 8 1–5 3.9 (1.0) 4.1 (3.3–4.8) 3436 (2.9%) 0.95
Instrumental social support 4 1–5 3.8 (1.2) 4.0 (3.0–5.0) 2097 (1.8%) 0.95
Emotional social support 4 1–5 3.9 (1.0) 4.0 (3.3–4.8) 2695 (2.3%) 0.95
Perceived stress 10 10–50 23.7 (7.8) 23.0 (17.8–29.0) 7171 (6.1%) 0.91
Everyday discrimination 9 0–5 0.8 (0.8) 0.7 (0.1–1.2) 3939 (3.3%) 0.91
Discrimination in health care settings 7 1–5 1.6 (0.6) 1.4 (1.0–1.9) 2906 (2.5%) 0.86
Social cohesion 4 1–5 3.8 (0.7) 3.8 (3.5–4.3) 1894 (1.6%) 0.87
Neighborhood physical disorder 6 1–4 1.6 (0.6) 1.5 (1.2–2.0) 3438 (2.9%) 0.84
Neighborhood social disorder 7 1–4 1.6 (0.5) 1.6 (1.1–2.0) 6251 (5.3%) 0.87
Daily spiritual experiences 6 0–6 2.7 (1.3) 2.5 (1.7–3.8) 1349 (1.1%) 0.81
PANES—Walking and bicyclinga 5 1–4 2.2 (0.9) 2.0 (1.4–2.8) 5771 (4.9%) 0.78
PANES—Crime and safety 2 1–4 3.5 (0.7) 4.0 (3.0–4.0) 15,392 (13.0%) NCb
Open in a new tab

PANES physical activity and neighborhood environment scale; NC non-calculable.

aFor PANES items related to walking and bicycling, respondents who answered this item “does not apply to my neighborhood” were not included in score distributions and were not counted in non-response totals; an alternative scoring strategy to preserve observations is described the Supplementary Methods.

bAlpha is non-calculable with 2 sub-items.

Item non-response and scale score distributions among survey respondents

Item non-response was infrequent. Ten of the 13 SDOH survey scales and sub-scales had fewer than 5% missing data due to item non-response (Table 3). The PANES Crime and Safety scale had the highest proportion of item non-response at 13.0%. The Cohen Perceived Stress Scale, the Neighborhood Social Disorder Scale, and the single housing quality item also had greater than 5% item non-response at 6.1%, 5.3%, and 5.2% respectively. Score distributions for scales are described in Table 3. Item non-response for the single Religious Service Attendance item was 1.5% (N = 1825). A single-item indicator from the PANES instrument for neighborhood residential density/housing type had 2.1% item non-response (N = 2475). Item non-response for the categorical measures was 1.3% (N = 1542) for food insecurity, 3.1% (N = 3620) for housing instability, and 5.2% for housing quality problems (N = 6134). Among survey respondents, 13.5% had food insecurity, 2.7% had housing instability assessed as multiple address changes in 12 months, and 21.1% had housing quality problems such as bug infestation, mold, or lead pipes.

Item non-response varied most by educational attainment (Fig. 1), racial identity (Fig. 2), and survey language (Fig. 2). Using the Loneliness scale as an example, participants with less than a high school degree or equivalent had 8.6% item non-response for the Loneliness scale compared to 1.9% among participants with a college or advanced degree. Black, African, and African American participants and Hispanic/Latino/Spanish participants had 5.1% and 5.7% item non-response for the Loneliness scale, respectively, compared to 2.0% item non-response among White participants. Participants who took the SDOH survey in Spanish had 10.6% item non-response for the Loneliness scale compared to 2.4% among those who took it in English. Similar patterns were observed for scales with larger item non-response. For example, In the PANES Crime and Safety 2-item scale, participants with less than a high school degree or equivalent had 21.3% item non-response compared to 11.8% among participants with a college or advanced degree. Black, African, and African American participants and Hispanic/Latino/Spanish participants had 18.0% and 17.1% item non-response respectively compared to 12.1% among White participants. Participants who took the survey in Spanish had 25.6% item non-response for PANES Crime and Safety compared to 12.7% item non-response among those who took it in English. A description of absolute differences in item non-response for each scale by participant characteristics is provided in Supplementary Table 4.

Figure 1.

Figure 1

Open in a new tab

Percentage of participants with incalculable scores due to item non-response by educational attainment. Abbreviations: PANES, Physical Activity and Neighborhood Environment Scale; PNA, prefer not to answer. Data represent the percentage of SDOH survey respondents with an incalculable scale due to item non-response, by educational attainment. Participants who responded to the incorrect response set for the Religious Service Attendance item (N = 11,795) are flagged as ‘invalid’ in version 7 data; these respondents are not included in item non-response calculations.

Figure 2.

Figure 2

Open in a new tab

Percentage of participants with incalculable scores due to item non-response by Racial Identity and language (English and Spanish). Abbreviations: PANES, Physical Activity and Neighborhood Environment Scale; MENA, Middle Eastern or North African; NHPI, Native Hawaiian or Pacific Islander; PNA, prefer not to answer. Black: Black, African or African-American; Hispanic: Hispanic/Latino/Spanish. Data shaded in gray are suppressed due to small sample size < 20. Data represent the percentage of SDOH survey respondents with an incalculable scale due to item non-response, by Racial Identity and Language group. Participants who responded to the incorrect response set for the Religious Service Attendance item (N = 11,795) are flagged as ‘invalid’ in version 7 data; these respondents are not included in item non-response calculations.

Multivariable logistic regression models

The odds of item non-response for any scale predicted by multivariable logistic regression are shown in Fig. 3. Black, African, or African-American (OR 1.55, 95% CI 1.48–1.63) and Hispanic/Latino/Spanish (OR 1.59, 95% CI 1.51–1.68) participants had higher item non-response compared to White participants. Item non-response was higher amongst those with lower educational attainment compared to those with a college or advanced degree. Item non-response was also lower for those with higher income levels compared to those making less than $50,000 per year. Compared to an individual aged 50, participants aged 25 had less item non-response (OR 0.56, 95% CI 0.54–0.59) and participants aged 75 had greater item non-response (OR 2.60, 95% CI 2.53–2.67). Bisexual (OR 0.86, 95% CI 0.80–0.93) participants had lower odds of item non-response for any scale than straight participants.

Figure 3.

Figure 3

Open in a new tab

Odds of item non-response or incalculable score for any SDOH scale. Abbreviations: PNA, prefer not to answer.

Several patterns stood out across all models (Supplementary Figure 5). For most scales, patterns in item non-response were seen by racial identity, educational attainment, income level, and age. As an example, in the Perceived Stress scale, Black, African or African-American participants (OR 1.78, 95% CI 1.64–1.93) and Hispanic/Latino/Spanish participants (OR 1.70, 95% CI 1.55–1.87) had a higher odds of item non-response compared to White participants; participants with less than a high school education were 1.79 (95% CI 1.55–2.06) times more likely to have item non-response compared to participants with a college or advanced degree; participants with an income level of over $150,000 per year were 0.73 (95% CI 0.67–0.80) times as likely to have item non-response compared to participants with an income level of less than $50,000 per year; and participants aged 25 years old were 0.60 (95% CI 0.54–0.66) times as likely and participants aged 75 years old were 2.71 (95% CI 2.59–2.84) times more likely to have item non-response compared to an individual aged 50 years old.

The food insecurity item, housing instability item, PANES Crime and Safety items, and housing quality item had atypical patterns in item non-response compared to other scales. For food insecurity, both a typical 25-year-old participant (OR 1.34, 95% CI 1.15–1.57) and a typical 75-year-old (OR 1.56, 95% CI 1.41–1.73) participant had higher odds of item non-response when compared to a 50-year-old participant. In housing instability, all racial identities other than the “prefer not to answer or skip" group had significantly higher odds of item non-response than White participants. Transgender and non-binary participants had lower odds of item non-response (OR 0.40, 95% CI 0.17–0.93) than those who identified as a woman. Both bisexual (OR 0.74, 95% CI 0.58–0.95) and gay (OR 0.66, 95% CI 0.50–0.85) participants had lower odds of item non-response than straight participants. However, those who did not answer the sexual orientation question had higher odds of item non-response missingness (OR 1.33, 95% CI 1.09–1.63) than straight respondents. For the PANES Crime and Safety items, all non-White racial identity groups had higher odds of item non-response than White participants. Typically, participants living with at least one identified disability had slightly higher odds of item non-response than those without identified disabilities, but in the housing quality item, participants living with a disability had lower odds of item non-response than those without identified disabilities (OR 0.86, 95% CI 0.79–0.94).

Discussion

The SDOH survey demonstrated good to excellent internal consistency in measurement of several SDOH concepts and within multiple diverse population groups, including those who are underrepresented in biomedical research. Among those who submitted the survey, data collection was fairly complete with low item non-response (0.9% to 13.5% missing) for most scales. Item non-response varied by SDOH measure and by demographic categories, notably, with differences in item non-response by racial identity, educational attainment, and survey language. Taken together, the relatively complete data among survey respondents and the internal consistency of scales are promising for the use of the survey to better understand the role of the measured SDOHs in precision medicine research. Importantly, however, patterns of survey nonresponse and item non-response are documented in this report to assist the researcher with managing bias due to differential survey nonresponse patterns in the current version 7 release. We note that few participants (2.2%) completed the survey in Spanish, and additional data are needed to confirm reliability and generalizability for administering the survey in Spanish.

The SDOH survey has limitations. The survey was designed to elicit participants’ perceptions of their social environments, psychosocial connections and conditions of their daily lives that would be applicable to multiple research studies and across several complex models of disease or health promotion. The survey does not measure or explore structural social determinants of health inequities that detail “the mechanisms through which social hierarchies and social conditions are created”1. For example, the survey sought to provide measures of perceived discrimination, but was not designed to gather hypothesis-driven information from participants on structural racism or policies that lead to participants’ experiences of unequal treatment. To facilitate further assessment of structural social determinants of health, the All of Us Research Program will conduct geocoding, and plan future assessment of area-level and other contextual measures to provide insight into macro-level social conditions of relevance to precision medicine. Additionally, the expansive scope of social experiences that participants may perceive as important to the condition of their lives is considerable, and all relevant concepts could not feasibly be included in one survey. For example, the Task Force deferred capturing more detailed measures of wealth and acculturation, for potential future assessment in dedicated data collection efforts. We note that other All of Us surveys currently measure income, home ownership, education, employment status, health literacy, and other topics related to SDOH, which are also available in the version 7 release. Continued engagement with All of Us participants and scientists is warranted to develop the next set of surveys that deepen scientific understanding of social concepts that influence health.

While data on internal consistency and data completeness are presented in this report, additional tests of psychometric properties of scales in the All of Us cohort should continue to be assessed in hypothesis-driven research. Some surveys, including PANES, assess multiple constructs and may perform differently in hypothesis-driven research. Importantly, the SDOH survey remains in the field, and as of January 2024, 43.8% of those eligible have completed this survey. As the survey is fielded more completely, metrics on survey non-response and updates on survey performance must continue to be monitored and evaluated to gauge internal reliability and external generalizability of SDOH data.

In conclusion, the All of Us SDOH survey, developed through engagement with scientists and All of Us participant partners, has items and scales with strong psychometric properties that measure social experiences among a large, diverse participant population for the purpose of advancing precision medicine research. Additional work is needed to investigate the construct validity of some social concepts by geography and within specific groups. For participants who took the survey in Spanish, and for Native Hawaiian and Pacific Islander groups, scale internal consistency reliability may have varied compared to other groups with larger sample sizes. Future surveys should add to the breadth of concepts that can be explored in All of Us, and where available, researchers can compare All of Us data with other cohorts to enhance our understanding of social experiences relevant to precision medicine research.

Supplementary Information

Supplementary Information. (2.9MB, pdf)

Acknowledgements

We thank our colleagues, Brandy Mapes, Ashley Green and Chris Lord for providing their support and input throughout the demonstration project lifecycle. We thank Jun Qian for providing input on the project’s code review. We thank the DRC’s Research Support team for their help during implementation. We also thank the All of Us Science Committee, the All of Us Participant Provided Information (PPI) Committee and All of Us Steering Committee for their efforts evaluating and finalizing the approved demonstration projects. The All of Us Research Program would not be possible without the partnership of contributions made by its participants. See the Supplementary Note for a roster of past and present All of Us principal investigators. To learn more about the All of Us Research Program’s research data repository, please visit https://www.researchallofus.org/. The All of Us Research Program is supported by the National Institutes of Health, Office of the Director: Regional Medical Centers: 1 OT2 OD026549; 1 OT2 OD026554; 1 OT2 OD026557; 1 OT2 OD026556; 1 OT2 OD026550; 1 OT2 OD 026552; 1 OT2 OD026548; 1 OT2 OD026551; 1 OT2 OD026555; IAA#: AOD 16037; Federally Qualified Health Centers: 75N98019F01202.; Data and Research Center: 1 OT2 OD35404; Biobank: 1 U24 OD023121; The Participant Center: U24 OD023176; Participant Technology Systems Center: 1 OT2 OD030043; Community Partners: 1 OT2 OD025277; 3 OT2 OD025315; 1 OT2 OD025337; 1 OT2 OD025276. In addition, the All of Us Research Program would not be possible without the partnership of its participants.

Author contributions

R.C. and C.R.C. were involved in the project design, project supervision, and manuscript development. R.C. and C.R.C. took part in writing all sections of the manuscript. S.L. was involved in the project administration and management. A.R. oversaw the project development and management. D.G.S. developed the scoring guidelines for each measure, provided expert opinion on the psychometric constructs, and helped write the Methods section of the manuscript. T.R.L. and Q.C. helped with the methodology and project implementation, and both made contributions to the Methods section. C.A.G. helped conduct the literature review and write the Abstract. S.T., A.G., and E.D.Y. were involved in the data curation, analysis, and visualization and contributed to the Methods and Results section. M.L.C., C.S.F., R.A.H., A.J., C.L.M.J., P.K., F.A.M., H.S., and E.N.V. were part of the SDOH Task Force and took part in the survey development process, the project design, and the review process of the manuscript. All authors participated in the editing and review process of the manuscript. S.T., R.C., and C.R.C. led the manuscript writing and review process.

Data availability

The SDOH survey is publicly available via the All of Us Survey Explorer at https://www.researchallofus.org/data-tools/survey-explorer/. The data can be found in the All of Us Researcher Workbench at https://www.researchallofus.org/data-tools/workbench/. The SDOH survey questions, responses with answer concept ids, number, and percentages of participants who selected each response, along with bar charts showing the number of participants who chose each answer by the sex assigned at birth and age when the survey was taken, are publicly available via the All of Us Data Browser at https://databrowser.researchallofus.org/.

Competing interests

The authors declare no competing interests.

Footnotes

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

The online version contains supplementary material available at 10.1038/s41598-024-57410-6.

References

  • 1.Solar O, Irwin A. A Conceptual Framework for Action on the Social Determinants of Health (World Health Organization, 2010). https://apps.who.int/iris/handle/10665/44489. Accessed 9 March 2023.
  • 2.Braveman P, Egerter S, Williams DR. The social determinants of health: Coming of age. Annu. Rev Public Health. 2011;32(1):381–398. doi: 10.1146/annurev-publhealth-031210-101218. [DOI] [PubMed] [Google Scholar]
  • 3.Denny J, Rutter J, Goldstein D, et al. The “All of Us” Research Program. N. Engl. J. Med. 2019;381(7):668–676. doi: 10.1056/NEJMsr1809937. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Mapes BM, Foster CS, Kusnoor SV, et al. Diversity and inclusion for the All of Us research program: A scoping review. PLOS ONE. 2020;15(7):e0234962. doi: 10.1371/journal.pone.0234962. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Fry A, Littlejohns TJ, Sudlow C, et al. Comparison of sociodemographic and health-related characteristics of UK biobank participants with those of the general population. Am. J. Epidemiol. 2017;186(9):1026–1034. doi: 10.1093/aje/kwx246. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Gaziano JM, Concato J, Brophy M, et al. Million veteran program: A mega-biobank to study genetic influences on health and disease. J. Clin. Epidemiol. 2016;70:214–223. doi: 10.1016/j.jclinepi.2015.09.016. [DOI] [PubMed] [Google Scholar]
  • 7.Cox LA, Hwang S, Haines J, et al. Using the PhenX toolkit to select standard measurement protocols for your research study. Curr. Protoc. 2021;1(5):e149. doi: 10.1002/cpz1.149. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.All of Us Research Hub. Researcher Workbench. https://www.researchallofus.org/data-tools/workbench/. Accessed 10 March 2023.
  • 9.Braveman PA, Kumanyika S, Fielding J, et al. Health Disparities and health equity: The issue is justice. Am. J. Public Health. 2011;101(Suppl 1):S149–S155. doi: 10.2105/AJPH.2010.300062. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.National Academies of Sciences, Engineering, and Medicine, Health and Medicine Division, Board on Population Health and Public Health Practice . Committee on Community-based solutions to promote health equity in the United States. In: Baciu A, Negussie Y, Geller A, Weinstein JN, editors. Communities in Action: Pathways to Health Equity. National Academies Press; 2017. [PubMed] [Google Scholar]
  • 11.Lewis CC, Wellman R, Jones SMW, et al. Comparing the performance of two social risk screening tools in a vulnerable subpopulation. J. Fam. Med. Prim. Care. 2020;9(9):5026–5034. doi: 10.4103/jfmpc.jfmpc_650_20. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Healthy People 2020. Office of Disease Prevention and Health Promotion, U.S. Department of Health and Human Services. Social Determinants of Health. HealthyPeople.gov. 2022. https://wayback.archive-it.org/5774/20220413203948/. https://www.healthypeople.gov/2020/topics-objectives/topic/social-determinants-of-health#top. Accessed 10 March 2023.
  • 13.Robert Wood Johnson Foundation (RWJF). Commission to Build a Healthier America. 2009. http://www.commissiononhealth.org/AboutUs.aspx. Accessed 10 March 2023.
  • 14.National Institute of Health All of Us Research Program. Participant Partners. All of Us Research Program | NIH. 2022. https://allofus.nih.gov/about/who-we-are/all-us-participant-partners. Accessed 9 May 2023.
  • 15.Cronin RM, Jerome RN, Mapes B, et al. Development of the initial surveys for the All of Us Research Program. Epidemiol. Camb. Mass. 2019;30(4):597–608. doi: 10.1097/EDE.0000000000001028. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Mujahid MS, Diez Roux AV, Morenoff JD, Raghunathan T. Assessing the measurement properties of neighborhood scales: From psychometrics to ecometrics. Am. J. Epidemiol. 2007;165(8):858–867. doi: 10.1093/aje/kwm040. [DOI] [PubMed] [Google Scholar]
  • 17.Moser A, Stuck AE, Silliman RA, Ganz PA, Clough-Gorr KM. The eight-item modified medical outcomes study social support survey: Psychometric evaluation showed excellent performance. J. Clin. Epidemiol. 2012;65(10):1107–1116. doi: 10.1016/j.jclinepi.2012.04.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Hays RD, DiMatteo MR. A short-form measure of loneliness. J. Pers. Assess. 1987;51(1):69–81. doi: 10.1207/s15327752jpa5101_6. [DOI] [PubMed] [Google Scholar]
  • 19.Williams DR. Race and health: Basic questions, emerging directions. Ann. Epidemiol. 1997;7(5):322–333. doi: 10.1016/S1047-2797(97)00051-3. [DOI] [PubMed] [Google Scholar]
  • 20.Williams, D. R. Everyday Discrimination Scale (Harvard University, 2023). https://scholar.harvard.edu/davidrwilliams/node/32397. Accessed 18 May 2023.
  • 21.Cohen S, Kamarck T, Mermelstein R. A global measure of perceived stress. J. Health Soc. Behav. 1983;24(4):385–396. doi: 10.2307/2136404. [DOI] [PubMed] [Google Scholar]
  • 22.Cohen, S. & Williamson, G. M. Perceived stress in a probability sample of the United States. In The Social Psychology of Health. The Claremont Symposium on Applied Social Psychology (eds Spacapan, S. & Oskamp, S.) 31-67 (Sage Publications, Inc, 1988).
  • 23.Masters KS. Brief multidimensional measure of religiousness/spirituality (BMMRS) In: Gellman MD, Turner JR, editors. Encyclopedia of Behavioral Medicine. Springer; 2013. pp. 267–269. [Google Scholar]
  • 24.Li S, Stampfer MJ, Williams DR, VanderWeele TJ. Association of religious service attendance with mortality among women. JAMA Intern. Med. 2016;176(6):777–785. doi: 10.1001/jamainternmed.2016.1615. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Regents of the University of California. CHIS 2018 Adult Questionnaire Version 1.53 Spanish 201. https://healthpolicy.ucla.edu/chis/design/Documents/2018%20Questionnaires%20and%20Topics%20List/09-11-19%20Updated/CHIS%202018%20Adult%20Spanish%20Questionnaire.pdf (2019).
  • 26.Hager ER, Quigg AM, Black MM, et al. Development and validity of a 2-item screen to identify families at risk for food insecurity. Pediatrics. 2010;126(1):e26–32. doi: 10.1542/peds.2009-3146. [DOI] [PubMed] [Google Scholar]
  • 27.Manchanda, R. & Gottlieb, L. Upstream Risks Screening Tool and Guide V2.6. https://www.aamc.org/media/25736/download (2015).
  • 28.Billioux A, Verlander K, Anthony S, Alley D. Standardized screening for health-related social needs in clinical settings: The accountable health communities screening tool. NAM Perspect. 2017 doi: 10.31478/201705b. [DOI] [Google Scholar]
  • 29.Ross CE, Mirowsky J. Neighborhood disadvantage, disorder, and health. J. Health Soc. Behav. 2001;42(3):258–276. doi: 10.2307/3090214. [DOI] [PubMed] [Google Scholar]
  • 30.Sallis JF, Kerr J, Carlson JA, et al. Evaluating a brief self-report measure of neighborhood environments for physical activity research and surveillance: Physical activity neighborhood environment scale (PANES) J. Phys. Act Health. 2010;7(4):533–540. doi: 10.1123/jpah.7.4.533. [DOI] [PubMed] [Google Scholar]
  • 31.Sallis, J. F. PANES: Physical Activity Neighborhood Environment Survey (also known as: IPS/IPAQ Environmental Survey Module, Self-Administered (original version). James F. Sallis PhD. https://www.drjimsallis.com/_files/ugd/a56315_068b49350b4a42208780d145e10d8ecc.pdf. Accessed 24 May 2023.
  • 32.Peek ME, Nunez-Smith M, Drum M, Lewis TT. Adapting the everyday discrimination scale to medical settings: Reliability and validity testing in a sample of African American patients. Ethn. Dis. 2011;21(4):502–509. [PMC free article] [PubMed] [Google Scholar]
  • 33.All of Us Research Program. Social Determinants of Health (SDOH) Survey-English. 2021. https://www.researchallofus.org/wp-content/themes/research-hub-wordpress-theme/media/surveys/SDOH_Survey_English.pdf. Accessed 10 May 2023.
  • 34.Earls FJ, Brooks-Gunn J, Raudenbush SW, Sampson RJ. Brooks-Gunn SW Raudenbush RJ Sampson 1999 Project on Human Development in Chicago Neighborhoods: Community Survey, 1994–1995. Verlag Herder; 1999. [Google Scholar]
  • 35.Sampson RJ, Raudenbush SW, Earls F. Neighborhoods and violent crime: A multilevel study of collective efficacy. Science. 1997;277(5328):918–924. doi: 10.1126/science.277.5328.918. [DOI] [PubMed] [Google Scholar]
  • 36.Taylor TR, Kamarck TW, Shiffman S. Validation of the Detroit area study discrimination scale in a community sample of older African American adults: The Pittsburgh healthy heart project. Int. J. Behav. Med. 2004;11(2):88–94. doi: 10.1207/s15327558ijbm1102_4. [DOI] [PubMed] [Google Scholar]
  • 37.Fetzer Institute NI on AWG. Multidimensional Measurement of Religiousness, Spirituality for Use in Health Research. A Report of the Fetzer Institute/National Institute on Aging Working Group. https://fetzer.org/sites/default/files/images/resources/attachment/%5Bcurrent-date%3Atiny%5D/Multidimensional_Measurement_of_Religousness_Spirituality.pdf (1999).
  • 38.Radandt NE, Corbridge T, Johnson DB, Kim AS, Scott JM, Coldwell SE. Validation of a 2-item food security screening tool in a dental setting. J. Dent. Child. 2018;85(3):114–119. [PMC free article] [PubMed] [Google Scholar]
  • 39.Ash AS, Mick EO, Ellis RP, Kiefe CI, Allison JJ, Clark MA. Social determinants of health in managed care payment formulas. JAMA Intern. Med. 2017;177(10):1424–1430. doi: 10.1001/jamainternmed.2017.3317. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Sallis JF, Bowles HR, Bauman A, et al. Neighborhood environments and physical activity among adults in 11 countries. Am. J. Prev. Med. 2009;36(6):484–490. doi: 10.1016/j.amepre.2009.01.031. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary Information. (2.9MB, pdf)

Data Availability Statement

The SDOH survey is publicly available via the All of Us Survey Explorer at https://www.researchallofus.org/data-tools/survey-explorer/. The data can be found in the All of Us Researcher Workbench at https://www.researchallofus.org/data-tools/workbench/. The SDOH survey questions, responses with answer concept ids, number, and percentages of participants who selected each response, along with bar charts showing the number of participants who chose each answer by the sex assigned at birth and age when the survey was taken, are publicly available via the All of Us Data Browser at https://databrowser.researchallofus.org/.

Articles from Scientific Reports are provided here courtesy of Nature Publishing Group

RESOURCES