National Academy of Medicine Social and Behavioral Measures: Associations With Self-Reported Health

Abstract

Introduction

Social and behavioral factors play important roles in physical and mental health; however, they are not routinely assessed in the healthcare system. A brief panel of measures of social and behavioral determinants of health (SBDs) were recommended in a National Academy of Medicine report for use in electronic health records. Initial testing of the panel established feasibility of use and robustness of the measures. This study evaluates their convergent and divergent validity in relation to self-reported physical and mental health and social desirability bias.

Methods

Adults, aged ≥18 years, were recruited through Qualtrics online panel survey platform in 2015 (data analyzed in 2015–2016). Participants completed the: (1) panel of SBD measures; (2) 12-Item Short Form Health Survey to assess associations with global physical and mental health; and (3) Marlow–Crowne Social Desirability scale short form to assess whether social desirability influenced associations between SBD measures and self-reported health.

Results

The sample included 513 participants (mean age, 47.9 [SD=14.2] years; 65.5% female). Several SBD domain measures were associated with physical and mental health. Adjusting for age, poorer physical and mental health were observed among participants reporting higher levels of financial resource strain, stress, depression, physical inactivity, current tobacco use, and a positive score for intimate partner violence. These associations remained significant after adjustment for social desirability bias.

Conclusions

SBD domains were associated with global measures of physical and mental health, and were not impacted by social desirability bias. The panel of SBD measures should now be tested in clinical settings.

INTRODUCTION

Healthcare delivery systems traditionally have focused on the biomedical treatment of disease and paid relatively little attention to social and behavioral factors that influence disease processes.¹ Given that social, environmental, and health-related behaviors account for at least half of premature deaths,^2–4 health systems and providers financially responsible for maintaining the health of populations, as a result of fixed payment reimbursement, are now devoting more attention to addressing these more root causes of health. Translating this growing interest into clinical practice will require new tools that improve the integration of social and behavioral care and medical care delivery. These tools will need to include standardized, evidence-based assessments of social and behavioral determinants of health (SBDs).

In 2013, the National Academy of Medicine (NAM), formerly known as the Institute of Medicine, convened an expert panel to identify a set of validated self-report measures to capture the most important SBDs of morbidity and mortality.⁵ The questions, which could be integrated with a healthcare system’s electronic health record, were selected by consensus from committee members representing healthcare services, informatics, and social and behavioral sciences. The final set of items was selected based on six criteria, including clinical significance and strength of the empirical evidence linking a given measure to health. The items spanned 12 domains, including race and ethnicity, education, financial resource strain, stress, depression, physical activity, tobacco use, alcohol use, social connection or isolation, intimate partner violence, and geocodable residential address.

The NAM committee’s report provided an important foundation for social and behavioral needs screening to support better clinical care and enable new discoveries. Each measure included in the recommended panel had previously been validated and shown to relate to health. However, the overall panel of measures needs further validation. The authors report here on the second stage of these tests. The first phase established that individuals could understand and complete the question panel in fewer than 5 minutes and with few omissions. Responses were stable over a period of 3 weeks and were not affected by question order.⁶ The goal of this second phase of research is to replicate key analyses in a new, independent sample and to examine convergent and divergent validity, including tests of whether responses relate to self-reported measures of physical and mental health and whether these associations are confounded by social desirability bias.

METHODS

A Qualtrics research panel of U.S. adults aged ≥18 years (www.qualtrics.com) was recruited to participate in the survey. Qualtrics research panel is an online survey platform available to researchers to facilitate participant recruitment and online data collection. The survey was available to Qualtrics research participants using an online platform. In total, 1,782 were contacted to participate, yielding 513 completers (28.8%). Each participant completed the NAM panel of SBD measures along with supplemental measures intended to evaluate its validity. Participants were paid $1 for survey completion. This study was approved by the IRB at the University of California, San Francisco, and written informed consent was obtained for each study participant.

Measures

Respondents reported their gender and age. Because this information is routinely recorded in health records, these items were not included in the SBD panel.

The NAM SBD panel consists of 25 items, representing the following 12 psychosocial constructs: race and ethnicity (two questions), education (two questions), financial resource strain (one question), stress (one question), depression (two questions), physical activity (two questions), tobacco use (two questions), alcohol use (three questions), social connection or isolation (five questions), intimate partner violence (four questions), and geocodable residential address to yield Census tract median income. Residential address was not included in this study to limit the amount of identifiable information obtained from the study sample. Census tract median income was also excluded as it relies on residential address.

Consistent with the first study evaluating the NAM measures, questions were modified from the original sources to conform with survey design best practices.⁶ The full questionnaire is available in Giuse et al.⁶

If a question was left blank, participants were probed with a follow-up question inquiring why the item was left blank. Possible answers included I do not wish to answer, I do not understand the question, and other. Participants who chose other were asked another follow-up question asking them to state why they did not answer using an open text format. This strategy applied to all SBD questions except for those about intimate partner violence.

The authors calculated questionnaire completion time using Qualtrics panel data. Completion time values > or <3 SDs from the mean were excluded from the calculations, which led to loss of 1.6% of the sample.

To assess discriminant validity (i.e., demonstrating that the scale is not simply capturing response tendencies like giving more socially-desirable responses) participants also completed the short form of the Marlow–Crowne Social Desirability (MCSD) Scale.⁷ The original MCSD scale has 33 items that ask the respondent to agree or disagree (true/false) with statements that either are highly desirable but unlikely to be true (e.g., I am always courteous, even to people who are disagreeable) or are common but unflattering tendencies (e.g., I am sometimes irritated by people who ask favors of me).⁸ Agreement with the former and disagreement with the latter are indicative of a tendency to portray oneself in a positive light and give more–socially desirable responses. The MCSD Scale has been used extensively in personality research and in psychometric evaluations.⁹ The authors used the 13-item short version of the scale, which has been shown to be a valid substitute for the long form version.⁷ Higher scores indicate greater levels of social desirability. There were 29 missing responses, resulting in MCSD scores on 485 of 513 participants (94.5%).

The 12-Item Short Form Health Survey (SF-12) survey is a well-validated self-report measure of physical and emotional health¹⁰ and allowed the authors to test whether the individual SBD measures were related to these indicators of global health status. The survey is composed of categorical (yes/no) and Likert-type scale response items that assess limitations in physical activity, physical role functioning, pain, vitality, social functioning, and overall subjective health. The SF-12 yields two composite scores: mental health and physical health. These scores are presented as t-scores with a mean of 50 and a SD of 10 in data from the general U.S. population. Higher scores on the composite measures are indicative of better subjective health. Missing responses resulted in available composites scores for 508 of 513 participants (99.0%).

Statistical Analysis

Descriptive statistics are presented as means with SDs or as percentages as appropriate. Pearson moment correlations were used to investigate associations between continuous variables, and independent t-tests and ANOVA were used to test group differences. Scores and classifications for depression (Patient Health Questionnaire-2 score), physical activity (Exercise Vital Sign), tobacco use, alcohol consumption (Alcohol Use Disorders Identification Test, Brief Screen score), social connection or isolation (Social Network Index), and intimate partner violence (Humiliation, Afraid, Rape, Kick questionnaire form) were calculated as described previously.⁶ ANCOVA, adjusting for age, was used to test differences in physical and mental health composite scores by SBD domains with Bonferroni corrected post hoc comparisons. To test the influence of social desirability, follow-up analyses additionally adjusted for MCSD scores. All statistical significance tests were two sided and significance was considered at p<0.05. All data were collected in 2015 and analyzed between 2015 and 2016. Statistical analyses were performed using SPSS, version 23.

RESULTS

Data were collected on 513 eligible participants using the Qualtrics research survey platform. The study sample was composed primarily of female adults (n=336, 65.5%) in midlife (mean age, 47.9 years; SD=14.2 years; range, 18–81 years). Table 1 provides descriptive data on the distribution of responses for each of the SBD domains. SF-12 composite scores for physical (mean, 46.7; SD=9.4) and mental health (mean, 46.7; SD=10.8) were positively correlated (r =0.33, p<0.001). Age was significantly associated with poorer self-reported physical health (r =−0.19, p<0.001) and better self-reported mental health (r =0.22, p<0.001). By contrast, there were no statistically significant differences between men and women on self-reported physical (t[506]= −0.35, p=0.73) or mental health (t[506]=0.21, p=0.98).

Table 1.

Participant Responses by National Academy of Medicine Panel Measure

Characteristics	n/N (%)
Race
White	408/512 (79.7)
Black	48/512 (9.4)
Other	44/512 (8.6)
Two or more races	12/512 (2.3)
Ethnicity
No, not Hispanic, Latino, or Spanish origin	479/513 (93.4)
Yes, Hispanic, Latino, or Spanish origin	34/513 (6.6)
Highest level of school
1–16 years (Elementary/High school/College)	430/513 (83.8)
17+ years (Graduate/Professional school)	83/513 (16.2)
Highest degree earned
Less than high school, high school diploma, GED	194/512 (37.9)
Vocational certificate or Associate’s degree	126/512 (24.6)
Bachelor’s degree	130/512 (25.3)
Master’s degree	50/512 (9.8)
Doctorate or Professional degree	12/512 (2.3)
Financial resource strain
Not hard at all	209/513 (40.7)
Somewhat hard or Very hard	304/513 (59.3)
Stress
Not at all	77/513 (15.0)
A little bit	179/513 (34.9)
Somewhat	121/513 (23.6)
Quite a bit or Very much	136/513 (26.5)
Depression (PHQ-2 score)
Negative screen (<3)	411/512 (80.3)
Positive screen (≥3)	101/512 (19.7)
Physical activity (EVS classification)
Inactive	118/513 (23.0)
Insufficiently active	227/513 (44.2)
Sufficiently active	168/513 (32.7)
Tobacco use
Never smoker	266/512 (52.0)
Former smoker	98/512 (19.1)
Current every day smoker or Current some day smoker	148/512 (28.9)
Alcohol use (AUDIT-score)
Negative screen	353/510 (69.2)
Positive screen	157/510 (30.8)
Social connection or isolation
Not isolated	37/512 (7.2)
Somewhat isolated	97/512 (18.9)
Very isolated	137/512 (26.8)
Most isolated	241/512 (47.1)
Intimate partner violence (HARK score)
Negative screen (<1)	434/510 (85.1)
Positive screen (≥1)	76/510 (14.9)

SF, Short Form; GED, General Education Diploma; PHQ, Patient Health Questionnaire; EVS, Exercise Vital Sign; AUDIT, Alcohol Use Disorders Identification Test; HARK, Humiliation, Afraid, Rape, Kick

Replicating prior findings,⁶ the SBD measure was completed quickly (median, 2.93 minutes; SD=4.88) and with few omissions. Overall, only 12 SBD questions (0.1% of the data) were left blank; one participant omitted two items, one depression item and one question related to intimate partner violence.

All the individual measures had been shown in prior research to be related to health outcomes. The authors tested whether those associations would hold when brief measures were included as part of a comprehensive panel. As shown in Table 2, neither the physical health nor the mental health composite scores on the SF-12 differed across sociodemographic groups (race, ethnicity, or education) or by alcohol use or social connectedness. Both health measures were, however, significantly related to a number of the psychosocial measures included in the panel. After adjusting for age, poorer physical and mental health were observed among individuals reporting higher levels of financial resource strain, stress, depression, physical inactivity, current tobacco use, and those with a positive score for intimate partner violence.

Table 2.

Differences in SF-12 Derived Physical and Mental Health Composites (T Scores) by National Academy of Medicine Panel Measures

Characteristics	Physical health composite (Adj. Mean, SE)a	p-value	R2	Mental health composite (Adj. Mean, SE)a	p-value	R2
Race		0.724	0.03		0.789	0.04
White	47.0 (0.5)			46.5 (0.5)
Black	45.7 (1.4)			48.1 (1.5)
Other	46.8 (1.4)			47.1 (1.6)
Two or more races	44.7 (2.7)			47.3 (3.1)
Ethnicity		0.642	0.03		0.131	0.05
No, not Hispanic, Latino, or Spanish origin	46.8 (0.4)			46.9 (0.5)
Yes, Hispanic, Latino, or Spanish origin	46.0 (1.6)			44.0 (1.8)
Highest level of school		0.832	0.03		0.508	0.04
1–16 years (Elementary/High school/College)	46.7 (0.5)			46.6 (0.5)
17+ years (Graduate/Professional school)	46.9 (1.0)			47.4 (1.2)
Highest degree earned		0.352	0.03		0.388	0.05
Less than high school, high school diploma	46.6 (0.7)			45.7 (0.8)
GED
Vocational certificate or Associate’s degree	46.6 (0.8)			47.4 (0.9)
Bachelor’s degree	47.6 (0.8)			47.2 (0.9)
Master’s degree	46.8 (1.3)			48.1 (1.5)
Doctorate or Professional degree	42.0 (2.7)			44.0 (3.0)
Financial resource strain		<0.001	0.08		<0.001	0.13
Not hard at all	49.3 (0.6)			50.5 (0.7)
Somewhat hard or Very hard	45.0 (0.5)			44.1 (0.6)
Stress		<0.001	0.12		<0.001	0.39
Not at all	50.6 (1.0)			55.7 (1.0)
A little bit	49.0 (0.7)			51.3 (0.6)
Somewhat	45.6 (0.8)			44.9 (0.8)
Quite a bit or Very much	42.6 (0.8)			37.3 (0.7)
Depression (PHQ-2 score)		<0.001	0.08		<0.001	0.37
Negative screen (<3)	47.8 (0.4)			49.8 (0.4)
Positive screen (≥3)	42.5 (0.9)			34.2 (0.9)
Physical activity (EVS classification)		<0.001	0.11		<0.001	0.11
Inactive	41.3 (0.8)			42.1 (1.0)
Insufficiently active	47.7 (0.6)			46.7 (0.7)
Sufficiently active	49.1 (0.7)			49.8 (0.8)
Tobacco use		<0.001	0.07		<0.001	0.08
Never smoker	48.1 (0.6)			48.7 (0.6)
Former smoker	47.3 (0.9)			45.5 (1.1)
Current every day smoker or Current some day smoker	43.9 (0.8)			43.8 (0.9)
Alcohol use (AUDIT-score)		0.593	0.03		0.057	0.05
Negative screen	46.9 (0.5)			47.3 (0.6)
Positive screen	46.4 (0.7)			45.4 (0.8)
Social connection or isolation		0.859	0.03		0.292	0.05
Not isolated	46.6 (0.6)			45.9 (0.7)
Somewhat isolated	46.6 (0.8)			46.7 (0.9)
Very isolated	47.5 (0.9)			48.3 (1.1)
Most isolated	46.6 (1.5)			47.4 (1.7)
Intimate partner violence (HARK score)		0.001	0.06		<0.001	0.08
Negative screen (<1)	47.4 (0.4)			47.5 (0.5)
Positive screen (≥1)	42.8 (1.1)			42.1 (1.2)

Note: Boldface indicates statistical significance (p<0.05)

^aAdjusted for age

SF, Short Form; GED, General Education Diploma; PHQ, Patient Health Questionnaire; EVS, Exercise Vital Sign; AUDIT, Alcohol Use Disorders Identification Test; HARK, Humiliation, Afraid, Rape, Kick

Having found that many of the panel measures were significantly related to global physical and mental health, the authors then tested whether these associations could be accounted for by confounding of the measures with social desirability. Individuals who are more likely to report socially desirable aspects of themselves and less likely to report unfavorable characteristics or behaviors may also be more likely to report better physical and mental health and under-report health problems. To assess this possibility, the authors first tested the bivariate associations between the MCSD Scale and each of the NAM panel of measures and with each of the two health outcomes. Higher MCSD scores were correlated with reporting better health on both SF-12 measures (physical health composite: r =0.19, p<0.001; mental health composite: r =0.43, p<0.001). In addition, MCSD scores were significantly higher among those reporting lower levels of financial strain (t[483]= −3.55, p<0.001), stress (F[3, 481]=25.96, p<0.001), and depression (t[482]=5.40, p<0.001), and an absence of intimate partner violence (t[480]=4.56, p<0.001).

The key test of confounding is not simply the association of social desirability with self-reported health or the psychosocial measures, but whether the association between the measures in the SBD panel and the health indicators are spurious and become non-significant after controlling for social desirability. As reported in Table 3, although social desirability was correlated with several SBD measures and with self-rated health, statistical adjustment for MCSD scores did not appreciably affect associations between SBD variables and SF-12 composites (Table 3).

Table 3.

Differences in SF-12 Derived Physical and Mental Health Composites (T-Scores) by National Academy of Medicine Panel Measures

Characteristics	Physical health composite (Adj Mean, SE)a	p-value	R2	Mental health composite (Adj Mean, SE)a	p-value	R2
Race		0.731	0.10		0.670	0.19
White	46.99 (0.46)			46.60 (0.50)
Black	45.65 (1.32)			48.52 (1.44)
Other	46.86 (1.41)			46.98 (1.54)
Two or more races	45.21 (2.58)			48.32 (2.82)
Ethnicity		0.665	0.10		0.146	0.19
No, not Hispanic, Latino, or Spanish origin	46.81 (0.42)			47.01 (0.46)
Yes, Hispanic, Latino, or Spanish origin	46.08 (1.62)			44.36 (1.76)
Highest level of school		0.676	0.10		0.389	0.19
1–16 years (Elementary/High school/College)	46.68 (0.45)			46.67 (0.48)
17+ years (Graduate/Professional school)	47.14 (0.99)			47.69 (1.08)
Highest degree earned		0.272	0.10		0.293	0.20
Less than high school, high school diploma, GED	46.36 (0.67)			45.72 (0.73)
Vocational certificate or Associate’s degree	46.32 (0.81)			47.07 (0.88)
Bachelor’s degree	48.05 (0.80)			47.59 (0.88)
Master’s degree	47.16 (1.28)			48.69 (1.39)
Doctorate or Professional degree	43.29 (2.57)			46.41 (2.80)
Financial resource strain		<0.001	0.13		<0.001	0.25
Not hard at all	48.84 (0.63)			50.13 (0.67)
Somewhat hard or Very hard	45.37 (0.52)			44.63 (0.55)
Stress		<0.001	0.15		<0.001	0.45
Not at all	49.80 (1.07)			55.03 (0.98)
A little bit	48.52 (0.67)			50.89 (0.62)
Somewhat	46.12 (0.84)			45.78 (0.77)
Quite a bit or Very much	43.35 (0.79)			37.99 (0.73)
Depression (PHQ-2 score)		<0.001	0.12		<0.001	0.45
Negative screen (<3)	47.52 (0.45)			49.64 (0.41)
Positive screen (≥3)	43.70 (0.92)			35.51 (0.84)
Physical activity (EVS classification)		<0.001	0.16		<0.001	0.24
Inactive	41.94 (0.84)			42.57 (0.92)
Insufficiently active	47.58 (0.58)			46.89 (0.64)
Sufficiently active	48.93 (0.69)			49.70 (0.76)
Tobacco use		<0.001	0.12		0.001	0.22
Never smoker	47.95 (0.56)			48.38 (0.61)
Former smoker	47.29 (0.94)			46.21 (1.02)
Current every day smoker or Current some day smoker	44.28 (0.74)			44.49 (0.81)
Alcohol use (AUDIT-score)		0.760	0.10		0.110	0.19
Negative screen	46.86 (0.49)			47.30 (0.53)
Positive screen	46.59 (0.73)			45.77 (0.80)
Social connection or isolation		0.791	0.10		0.159	0.20
Not isolated	46.05 (1.51)			47.29 (1.64)
Somewhat isolated	47.57 (0.93)			48.80 (1.01)
Very isolated	46.57 (0.78)			46.62 (0.84)
Most isolated	46.73 (0.60)			46.09 (0.65)
Intimate partner violence (HARK score)		0.003	0.11		0.015	0.20
Negative screen (<1)	47.27 (0.44)			47.30 (0.48)
Positive screen (≥1)	43.77 (1.08)			44.17 (1.18)

Note: Boldface indicates statistical significance (p<0.05).

^aAdjusted for age and Marlow Crowne Social Desirability scores.

SF, Short Form; GED, General Education Diploma; PHQ, Patient Health Questionnaire; EVS, Exercise Vital Sign; AUDIT, Alcohol Use Disorders Identification Test; HARK, Humiliation, Afraid, Rape, Kick

DISCUSSION

This study provides new evidence on feasibility, reliability, and validity of the panel of psychosocial “vital signs” recommended by an NAM committee for use in electronic health records. Findings from this new, independent sample replicated the previous findings regarding feasibility and reliability⁶ as participants in this study also completed the panel of items quickly (fewer than 3 minutes) and left a negligible number of items unanswered. The current study went beyond the prior tests by also examining associations between the social and behavioral measures included in the NAM panel and self-reported global mental and physical health.

Consistent with the literature demonstrating associations between social and behavioral factors and health,¹¹ the authors found that several of the measures accounted for significant variability in SF-12 physical and mental health composite scores. Specifically, greater reports of financial strain, stress, depressive symptoms, physical inactivity, current tobacco use, and intimate partner violence were associated with both poorer self-reported physical and mental health. These associations remained significant when adjusted for age and social desirability. Given that the individual measures were selected partly on the basis of their having been related to some health outcomes, it is not surprising to find that most of the SBD domains mapped onto self-reported measures of physical and mental health; the current work confirms that the combination of these individual measures into a more comprehensive panel does not affect those associations.

Limitations

The major limitation of this work is the use of a web-based tool for recruitment of participants. Like other such tools, the Qualtrics research survey platform aims to yield nationally representative samples, but self-selection of participants may occur. For example, though not statistically significant, participants in this sample who reported having a doctoral or professional degree reported poorer physical health than did participants with less education. The nature of the Qualtrics task (completing surveys for pay) may pull for individuals who are not experiencing the health benefits typically correlated with higher education. Additionally, women and Caucasians were over-represented in this sample. Although the key findings are unlikely to change, future work should repeat the tests of reliability and validity in settings where respondents will mirror the population for whom clinical care is provided. Finally, Census tract median income, which was included in the NAM panel of SBD measures, was excluded from this survey. This exclusion was made because it relies on residential address and the authors wanted to limit the identifiable information obtained from participants. As such, these analyses are restricted largely to psychosocial and behavioral variables.

CONCLUSIONS

The evidence provided here, along with findings from the prior evaluation of the SBD panel recommended by the NAM, supports the use of the panel in clinical settings. Systematic incorporation of this information into electronic health records will help individual clinicians and health systems identify patients at elevated risk of health problems associated with social determinants of health.⁵

The inclusion of the panel of measures in electronic health records will also advance efforts to develop precision medicine. Though much of the excitement around precision medicine involves innovations in measuring the various levels of biological analyses (e.g., genomic, proteomic, metabolomic) development of individualized treatment and prevention strategies will require expanding understanding of the determinants of health and developing strategies to integrate information on social and behavioral factors into care delivery.^12,13 The standardized, parsimonious panel of SBD measures recommended by the NAM committee could help advance this integration. Though more work will be needed to help the healthcare system overcome other barriers to implementation,¹⁴ the present study finds that collection of these SBD measures is feasible and can be clinically meaningful.