Methods Used to Evaluate the Health Effects of Social Policies: A Systematic Review

Abstract

Purpose of review:

Despite the growth of research on social policies and health in recent years, few studies have systematically summarized the methodological approaches used in this growing literature. This review characterizes the range of and trends in analytic methods used in studies of the health effects of US social policies published in leading health journals during January 2014-July 2024.

Recent findings:

Among the 117 studies reviewed, confounder-control approaches were the most commonly used method to assess health effects of social policies. Quasi-experimental methods were also frequently used, especially difference-in-differences designs. Heterogeneous subgroup effects were consistently assessed.

Summary:

Although there was frequent use of quasi-experimental designs that meet standards for rigorous evidence used to inform policymaking, many opportunities for improvement remain. We suggest improvements to data infrastructure, highlight less frequently studied policies as fruitful future research opportunities, and encourage researchers to implement quasi-experimental approaches best suited to identify causal estimates.

Introduction

Research on the health effects of social policies has grown dramatically in the last decade, with contributions from the fields of epidemiology, public health, economics, and other social sciences. Such studies examine how social policies related to education, income, wages, work-related benefits, housing assistance, and more shape population health and health inequities (1-6). Theoretical and empirical work increasingly support the idea that social policies often dictate the distribution of resources based on an individual’s or family’s social and economic characteristics, which then determine access to health-promoting opportunities and resources, as well as risk of exposure to harms (7-13). Practically speaking, the importance of this research is also demonstrated by calls from the US Congressional Budget Office (CBO), which publishes projections on the costs and benefits of any given policy. CBO states: “The literature is lacking on how policies that focus on nutrition, education, housing, and employment would affect the federal budget through their influences on people’s health [emphasis ours]. An evidence-based body of research would enable CBO to estimate the effects of those policies on the budget” (14).

As this body of research expands, studies are often inconsistent in their application of well-suited policy evaluation methods—in particular, quasi-experimental methods—to rigorously identify the effects of such social policies using observational data (15,16). These methods often rely on the potential outcomes framework to define a counterfactual scenario, including exploiting natural experiments. Their goal is to provide stronger causal inference on how policies influence health (17). While a range of methods—including descriptive epidemiologic surveillance, qualitative methods, or comparative analysis—can generate useful insights regarding how social policies shape health outcomes, governmental and non-governmental bodies like the CBO preferentially rely on more sophisticated analytic approaches to ensure rigor when recommending the allocations of billions of dollars of government funds (18,19).

Despite this growing interest and the changing empirical strategies available, few studies have systematically assessed the full range of methods used by investigators to examine the impact of social policies on health in recent years. Such an assessment of the state of the field would provide insight into remaining gaps in the literature and how research funders should direct limited resources. Previous reviews of the epidemiology literature have summarized the growth of, strengths of, and challenges to quasi-experimental methods—a particularly popular subset of methods that includes difference-in-differences, instrumental variables, and more (15,16,20,21). Reviews of the economics literature have documented the rapid growth of quasi-experimental methods in recent years (22,23). Others have assessed the literature on the health effects of a specific social policy (24,25) or summarized the findings on how social policies affect health (26). Yet others have provided methodological overviews of recent advances in quasi-experimental methods (27-29). There has been less attention to whether and how these methodological innovations have been taken up by public health researchers.

In this systematic review, we identified and characterized the methods used in studies on the health effects of US social policies published in the last 10 years in leading health journals. We focused on social policies most commonly examined in the literature that address social determinants of health like income, nutrition, and education, e.g., the Earned Income Tax Credit (EITC) and the Supplemental Nutrition Assistance Program (SNAP). We also documented key analytic features of these studies. This review of the methods used in studies of the health effects of US social policies thus characterizes this growing, active field and provides directions for future research.

Methods

During July-September 2024, we conducted a systematic review of studies investigating the effects of US social policies on health following the 2020 Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) systematic review guidelines (30).

Search Strategy and Screening

Authors co-developed the inclusion criteria and search terms, and one co-author searched PubMed on August 8, 2024. Our inclusion criteria were as follows: exposure of interest was a social policy in the US; outcome was health-related; published in English; published during 2014-2024 in one of 17 journals that had previously been identified as leading journals overall and within epidemiology that publish research on the health effects of social policies (21). When not limiting the search to these journals, we captured over 65,000 articles. Thus, we limited our search to these 17 journals to keep the review feasible while ensuring we focused on articles that represented the highest quality research on this topic. We included only original, empirical studies, excluding reviews or commentaries. We excluded articles that examined hypothetical social policies (e.g., policy simulations), randomized controlled trials of policy-like interventions (e.g., Moving to Opportunity), or court decisions, because the goal of this review was to summarize methods used to study health effects of social policies that have been implemented in the US. Further, studies on hypothetical policies and policy-like interventions likely use different methods than studies on actual policies since the conditions are different (i.e., hypothetical and experimental compared to real-world), and were thus not applicable for this review.

Policies were defined as “social” if they impacted the availability or the quality of socioeconomic resources associated with well-being, including income (e.g., tax credits, cash assistance), food assistance, housing assistance, and education, and were not explicitly health or healthcare policies (e.g., Medicaid). The authors used their knowledge of policies in these domains to generate a list of policy search terms; this was not intended to be inclusive of all US social policies, but rather those that are commonly analyzed in the literature. We did not list specific health outcomes in the search terms since we conducted our search in health journals only. This search strategy ensured the articles were relevant to health, but allowed us to capture a broader range of health outcomes than if we had searched for only specific terms. The titles, abstracts, and complete texts of all search results (1,459 articles) were exported into Covidence, an online systematic review software (31).

To determine whether an article was appropriate for inclusion, two co-authors first screened the title and abstract of the articles. Both co-authors screened 100 articles (6.9%) to ensure consistency and inter-rater reliability. With 100% screening agreement, the co-authors proceeded to screen the remaining 1,359 articles independently. We made no exclusions based on study quality, because we sought to characterize the range of methods used to study social policies and health. Based on the titles and abstracts, 143 articles were kept for full article review. Of these 143 articles, we excluded any remaining articles that were not original studies (n=3), did not analyze a social policy (n=21), or did not include a health outcome (n=2). The final sample size was 117 (1-6,32-142).

Data Extraction

We then extracted data from each study, using a standard form in Covidence to abstract the policies and health outcomes assessed, methods used (e.g., type of data, analytic methods, comparison group). Since some groups may be affected differently by policies, we also captured whether and how potential effect heterogeneity was examined, as this has been highlighted as a key analytic strength of policy evaluation studies (143). We recorded all analytic methods used within each study, since many articles used multiple analytical approaches. If a given method could be characterized by multiple available categories on our data extraction tool (e.g., two-way fixed effects was used to generate a difference-in-differences estimate while controlling for confounders), we characterized it only in relation to its overall approach to causal identification (i.e., in this example, differences-in-differences). See Appendix Table A1 for a brief description of each method. We did not collect detailed estimates of the policy effects or conduct assessments of the risk of bias. The aim of our review was not to report or meta-analyze estimated associations between social policies and health—especially given the heterogeneity in policies and health outcomes included—but rather to describe the state of the methods in the field on social policies and health.

Two co-authors piloted the extraction tool on the same two articles before full data extraction began. Based on this process, the co-authors made additional refinements to the extraction tool to ensure that we were extracting the data as originally intended and in similar ways to ensure consistency. The two co-authors then split the remaining 115 articles so that data were extracted from each of the remaining articles by only one person, with consultations for a second opinion when necessary.

In addition to the description of methods (Table A1), the Appendix also contains the search terms (Table A2), data extraction tool (Table A3), list of journals included in the search (Table A4), a brief description of the relevant social policies (Table A5), and the PRISMA flow chart of the search process (Figure A1). We also provide a full list of studies and their characteristics in Online Resource 1.

Data Cleaning and Analysis

We exported the final extraction data from Covidence and imported it into Stata 18.5 (StataCorp; College Station, TX). We revisited data from studies originally categorized as “Other” during the extraction process to recategorize them into existing or new categories when appropriate.

We tabulated the number and percent of included studies for each extracted data point described above. To visualize results, we tabulated the number of methods used each year (including multiple methods for each article when applicable), as well as the percent of included articles that used at least one quasi-experimental method each year across all journals. We included difference-in-differences, fixed effects, matching, interrupted time series, instrumental variables, synthetic control, and regression discontinuity as quasi-experimental methods, because they explicitly define a counterfactual scenario and leverage potentially exogenous variation in the policy exposure. Confounder-control, and paired sample t-tests were included as not quasi-experimental. Different disciplines may or may not categorize these methods as quasi-experimental, based on assessments of residual confounding and which tradeoffs are more important for appropriately answering the research question. We do not intend to rank these methods in terms of their ability to produce causal estimates, as other studies have already covered this topic in depth (e.g., 143). Finally, we also conducted a descriptive analysis of how studies measured an individual’s exposure to the policy, including examples.

Results

Although our search strategy included 17 journals, the 117 included studies were concentrated in four journals: the American Journal of Preventive Medicine (23.1%), Social Science and Medicine (21.4%), Health Affairs (17.1%), and the American Journal of Public Health (15.4%) (Table 1). No included studies were published in six of the included journals, though this was likely due to the focus of these journals on studies based outside the US (e.g., European Journal of Epidemiology) and review articles (e.g., Epidemiologic Reviews). Most studies examined the policy effects on adult health outcomes (58.1%), about a quarter examined the effects on child health (26.5%), and the rest examined effects on all ages (15.4%).

Table 1.

Characteristics of studies examining social policies and health in the United States published in selected English-language health journals from 2014-2024 (n=117).

Characteristic	N (%)
Year published
2014	6 (5.1)
2015	7 (6.0)
2016	7 (6.0)
2017	15 (12.8)
2018	7 (6.0)
2019	10 (8.5)
2020	14 (12.0)
2021	16 (13.7)
2022	21 (17.9)
2023	7 (6.0)
2024	7 (6.0)
Journal a
American Journal of Preventive Medicine	27 (23.1)
Social Science and Medicine	25 (21.4)
Health Affairs	20 (17.1)
American Journal of Public Health	18 (15.4)
American Journal of Epidemiology	9 (7.7)
Annals of Epidemiology	7 (6.0)
Journal of Epidemiology and Community Health	5 (4.3)
Other journals included in search strategy	6 (5.1)
Population of interest
Adults	68 (58.1)
Children	31 (26.5)
All ages	18 (15.4)
Type of health outcome data b
Survey	84 (71.8)
Administrative	23 (19.7)
Food purchases	6 (5.1)
Medical claims and health records	6 (5.1)
Analytic method(s) b
Confounder-control	45 (38.5)
Difference-in-differences	41 (35.0)
Heterogeneity-robust difference-in-differences	4 (3.4)
Fixed effects	23 (19.7)
Matching	15 (12.8)
Instrumental variables	11 (9.4)
Interrupted time series	4 (3.4)
Synthetic control	3 (2.6)
Randomization	1 (0.9)
Paired sample t-tests	1 (0.9)
Regression discontinuity	0 (0.0)
Approach to operationalizing policy exposure b
Imputed/proxy based on self-reported variables	62 (53.0)
Self-report program participation	40 (34.2)
Sampled from a known participating group	9 (7.7)
Linkage to administrative data	8 (6.8)
Type of health outcome(s) b
Physical health	64 (54.7)
Health behavior	49 (41.9)
Mental health	32 (27.4)
Healthcare access	17 (14.5)
Policy(ies) examined b
Work	43 (36.8)
Minimum wage	15 (12.8)
Paid sick leave	12 (10.3)
Unemployment insurance	10 (8.5)
Paid family and/or medical leave	7 (6.0)
Social Security retirement benefits	2 (1.7)
Workers’ compensation	1 (0.9)
Food	42 (35.9)
Supplemental Nutrition Assistance Program (SNAP)	29 (24.8)
Special Supplemental Nutrition Program for Women, Infants, and Children (WIC)	13 (11.1)
School meal financial support	3 (2.6)
School meal nutritional guidelines	3 (2.6)
Cash transfers	24 (20.5)
Earned Income Tax Credit (EITC)	12 (10.3)
Temporary Assistance for Needy Families (TANF)	7 (6.0)
Child Tax Credit (CTC)	5 (4.3)
Other cash transfers	3 (2.6)
Aid to Families with Dependent Children (AFDC)	1 (0.9)
Supplemental Security Income (SSI)	1 (0.9)
Housing	11 (9.4)
Rental assistance	6 (5.1)
Eviction	3 (2.6)
Neighborhood and community development	2 (1.7)
Education	5 (4.3)
Compulsory schooling laws	3 (2.6)
Head Start	1 (0.9)
School quality	1 (0.9)
Any public assistance	1 (0.9)
Examined heterogeneity in policy’s effect on outcome(s) Source(s) of heterogeneity examined b	64 (54.7)
Demographic (e.g., race/ethnicity, gender/sex, marital status, age)	52 (44.4)
Socioeconomic (education, income, employment)	20 (17.1)
Geography (state poverty rates, neighborhood disadvantage)	9 (7.7)
Health (insurance, food insecurity, health conditions, behaviors)	8 (6.8)

^aWe listed all journals for which there were ≥5 included studies. The remaining journals were grouped under “Other.” See Online Resource 1 for the full list of studies and their characteristics.

^bStudies could be categorized using one or more of the available response options. Thus, frequencies do not sum to 117 and percentages do not sum to 100. Percentages represent the percent of all 117 studies were observed to have the indicated characteristic.

Policies and Health Outcomes Analyzed

See Table 1 for sample characteristics; note that some percentages sum to >100% because several articles examined more than one policy, employed more than one method, etc. The articles assessed the effects of policies that we categorized into five broad categories based on the type of support provided or need that was targeted: work (36.8%), food (35.9%), cash transfers (N=20.5%), housing (9.4%), and education (4.3%). Since minimum wage policy targets work-specific earnings, it was categorized as a work policy. Similarly, SNAP benefits can only be used for food purchases, so we considered it a food assistance policy. Meanwhile, the EITC provides non-directed cash transfers (via tax credits); thus, we categorized it under cash transfers.

Studies on work policies mostly focused on minimum wage, paid leave, and unemployment benefits. SNAP and the Special Supplemental Assistance Program for Women, Infants, and Children (WIC) were the most frequently examined food policies, but there were also studies on the National School Lunch Program. The EITC and Temporary Assistance for Needy Families (TANF) were the most common cash transfer policies studied. Five cash transfer studies examined the Child Tax Credit (CTC), and three examined other cash transfer programs (e.g., an index of COVID-19-related cash transfers). Meanwhile, most studies of housing policies focused on rental assistance programs, including vouchers and tax credits, and a few studies examined eviction policy and neighborhood and community development. Finally, education policies were mostly comprised of compulsory schooling laws.

The type of health outcomes examined by these studies varied, and broadly included physical health (54.7%), health behaviors (41.9%), mental health (27.4%), and healthcare access and utilization (14.5%). Examples of physical health outcomes were days of poor physical health, mortality, hypertension, and low birth weight. Examples of health behaviors included food purchasing, diet, smoking, alcohol consumption, drug use, and vaccination. Examples of mental health included psychological distress, number of poor mental health days, and depression. Finally, examples of healthcare access included prenatal care visits, cancer screening rates, and health insurance status. To measure the health outcomes, a majority of studies drew from survey data (71.8%; e.g., references 2,113), fewer from administrative data (19.7%; e.g., references 34,35), and the fewest from food purchases (5.1%; e.g., references 88,102) and medical claims/health records (5.1%; e.g., references 33,77).

Most Commonly Used Methods

The most commonly used method in these studies was confounder-control (38.5%; e.g., 5,55), which we define as a method that primarily depends on conditioning on observed covariates to isolate the relationship between the exposure and health. Difference-in-differences models (including the newer heterogeneity-robust approaches) were the second most common (35.0%; e.g., 36,117), followed by fixed effects (19.7%; e.g., for geographic units in 69,119). Less widely used methods included matching (12.8%; e.g., 3,46), instrumental variable analysis (9.4%; e.g., 1,42), and interrupted time series (3.4%; e.g., 35,77). No study in our sample used a regression discontinuity design.

The frequency of use of each type of method had, in some cases, changed over time (Figure 1, Panel a). Confounder-control methods were prevalent across the study period. For most years, confounder-control was used at least 3 times, with the most usage being 10 times in 2017 and 7 times in 2022; however, it was used only one time in 2016 and 2023. Difference-in-differences study designs were also prevalent across the study period, and showed signs of increasing in more recent years (used 1-4 times per year during 2014-2019, and 6-10 times per year during 2020-2023). Although there has not been broader use of heterogeneity-robust difference-in-differences yet, these methods were used twice in both 2022 and 2023 (after being used 0 times from 2014-2021), suggesting entry of these methods into the field of social policies and health research. The use of fixed effects was relatively consistent (most years it was used 1-3 times, and four times in 2021). Synthetic control methods were not prevalent, only being used in studies published in 2015 and 2020. Instrumental variable use appeared to decline; they were used 1-2 times per year in 2014-2018, and 0-1 times per year in 2019-2024.

Figure 1. Methods used for social policy evaluations by year.

Panel a. Studies could be categorized using one or more of the methods. Thus, the number of studies does not sum to 117. Panel b. Studies were classified as using at least one quasi-experimental method if they used difference-in-differences, heterogeneity-robust difference-in-differences, fixed effects, synthetic controls, instrumental variables, regression discontinuity, interrupted time series and/or a matching method.

^a We conducted our literature search on August 8, 2024, meaning the data for 2024 represent a partial year.

In each year, at least 40% of all included articles used at least one quasi-experimental method (Figure 1, Panel b). There was no clear temporal pattern of quasi-experimental studies as a percentage of all studies.

Operationalization of Policy Exposure

How studies measured the policy exposure varied (Table 2). Most studies imputed policy exposure based on characteristics that proxied program participation, because the datasets lacked direct measures of program participation (53.0%). For example, in studies that examined food assistance and most cash transfer programs, participation was often proxied using a combination of variables to capture eligibility (such as gender, education, income, marital status, and/or household size; e.g., references 2,83). Because many policies differ across states, either because they are state policies or federal policies implemented by states, place of residence was often used as an indicator of policy exposure (e.g., 34,56). Because work-related policies are not means-tested but instead more relevant for working-age adults than children and older adults, these studies often relied on place of residence and respondent age as indicators of exposure (e.g., 43,131). These studies identified comparison or “placebo” groups based on similar identification strategies. For example, in one study on the health effects of minimum wage, the treated group was defined as working-age adults with a high school diploma or less, a group more likely to hold minimum wage jobs, and the placebo group was defined as working age adults with a college education or more (45).

Table 2.

Examples of Operationalizing Policy Exposures and Comparison Groups

Approach to exposure operationalization	Study	How exposure to policy was determined	Comparison groups
Self-reported program participation	Adams 2022	Considered exposed if survey participant reported receiving the 2021 Child Tax Credit	Compared participant survey responses from before and after the Child Tax Credit expansion
	Heflin et al. 2019	Considered exposed if survey participant reported receiving SNAP benefits	Compared participant survey responses based on SNAP participation
	Denary et al. 2021	Considered exposed if survey participant reported receiving rental assistance	Compared participant survey responses based on current rental assistance receipt
Imputed/proxy based on self-reported variables	Spencer et al. 2020	Considered exposed if living in a state when the policy was implemented	Compared mothers with a high school diploma/GED or less (treated) to mothers with more than a high school education (control)
	Gertner et al. 2019	All were considered exposed, but intensity of exposure varied by the value of the state or federal minimum wage in each state/year	Adults across states and years with varying levels of minimum wage
	Kenney et al. 2020	Considered exposed if outcome was measured after the enactment of the Healthy, Hunger Free Kids Act	Compared trends before and after the enactment of the Healthy, Hunger Free Kids Act
Sampled from a known participating group	Burgette et al. 2017	Exposed group was recruited directly from the Early Head Start program	Compared outcomes of community-matched parent-child dyads based on Early Head Start participation
	Duffy et al. 2024	Considered exposed if WIC Electronic Benefit Transfer was payment type used for groceries	Compared outcomes based on WIC participation
	Pollack et al. 2023	Exposed group was recruited directly from children enrolled in the Baltimore Regional Housing Partnership program	Compared outcomes of a matched sample based on enrollment in the housing program
Linkage to administrative data on program participation	Basu et al. 2016	Considered exposed if SNAP participation was confirmed by USDA in the FoodAPS survey	Compared outcomes based on SNAP participation and income eligibility
	Miller-Archie et al. 2019	Considered exposed if NYNY III housing records confirmed individual was placed in housing for at least 7 days	Compared outcomes based on placement in housing
	Edmunds et al. 2014	Considered exposed if the New York Pediatric Nutrition Surveillance System and the Pregnancy Surveillance System confirmed WIC receipt	Compared outcomes based on enrollment during different pregnancy trimesters

As another method of imputing policy exposure, many studies relied on self-reported program participation (34.2%; e.g., 6,93). The smallest number drew samples directly from a known participating group (such as participants recruited from program offices, 7.7%; e.g., 32,46) or identified participation exposure from linked administrative databases (6.8%; e.g., 72,130). Otherwise, linkages of administrative data on participation with health data were rare.

Over half of studies explored heterogeneous effects of policies, either by stratifying samples or by interacting the policy indicator with the characteristic of interest (54.7%). The grouping characteristics included demographic variables (race/ethnicity, gender/sex, marital status, age, whether the person was a parent, 44.4%; e.g., 76,124), socioeconomic variables (education, income, employment, 17.1%; e.g., 57,91), place-based factors (state poverty rates, neighborhood disadvantage, 7.7%; e.g., 49,53), and health-related variables (insurance, food security, health conditions, behaviors, 6.8%; e.g., 33,63).

Discussion

This systematic review summarized data from studies on the health effects of social policies published in the past 10 years in leading epidemiology, public health, and medical journals. We found that confounder-control methods were more frequently used than any of the quasi-experimental methods over the study period, although the frequency of use varied by year. Quasi-experimental methods were also frequently used across all years, and there was no clear increasing or decreasing trend over time. Difference-in-differences analyses were the most common quasi-experimental method used, and the use of heterogeneity-robust difference-in-differences approaches only started in the most recent years. Studies most frequently used data from surveys and administrative sources and tended to use proxies like education level or income to determine likelihood of program participation. The most commonly studied policies were SNAP and minimum wage, whereas housing and education policies were less frequently examined. Physical health and health behaviors were the most frequent outcomes examined, whereas studies that assessed healthcare outcomes were uncommon. Over half the studies included analyses on heterogeneous policy associations by subgroup.

This review captured the range of methods being used in analyses of social policies and health and highlighted recent use of quasi-experimental methods. A large percentage of social policy evaluations in the public health and medical literatures continue to employ methods that do not robustly support causal inference, which limits their potential to inform policy conversations and may limit their use by researchers in other disciplines that have rapidly increased their use of quasi-experimental methods, such as economics (22). While quasi-experimental methods are not always feasible and may not always be the most appropriate method for the research question, increased use of quasi-experimental methods would strengthen the literature on social policies and health; these methods allow researchers to draw stronger causal conclusions when data requirements are met and assumptions are satisfied, thereby maximizing translational impact. Other reviews of similar areas of literature have found that methods with weaker causal implications were still frequently used, but quasi-experimental methods had started to gain traction (15,145). For example, a recent review of general epidemiologic research (i.e., not specific to studies on the health effects of social policies) found an increase in the use of quasi-experimental methods in recent years, with almost zero quasi-experimental studies published in the 1990s, and 25 or more published each year during 2015-2021 (16). Public health training programs should endeavor to increase skills-based trainings in conducting such evaluations, drawing on the relevant literatures in epidemiology, economics, political science, and others.

This review also highlighted some of the methodological challenges of this research. Most studies relied on proxies or self-report for program participation, and only a small subset were able to confirm program participation through administrative linkages. Although some policies may have spillover effects for nonparticipants, in most cases, researchers aim to isolate policy effects on program participants. This is difficult to do without access to the appropriate data. Public policy evaluations in international settings often rely on data linkages across different government databases, but this is virtually impossible in the US given the legal and logistical hurdles in doing so. US researchers often rely on isolated linkages of program participation information with individual surveys—such as linkages of data from the National Health Interview Survey with data on housing assistance programs from the US Department of Housing and Urban Development (146)—but such linkages are rare. A recent report from the National Academies of Science, Engineering, and Medicine, highlighted the need for greater governmental investment in data infrastructure and data access to enable policy research to inform decision-making (18,147).

Our review also identified multiple substantive areas for future research and scholarship. Studies on SNAP were the most common, followed by minimum wage studies; together these represented more than one-third of all policies studied in the articles included in our review. Housing and education policies were less frequently examined, despite their established importance for health outcomes (148,149). Additional studies on the health effects of these policies in the US context would therefore be particularly important areas for future research. In addition, this research was concentrated in four of the journals we examined, American Journal of Preventive Medicine, Social Science & Medicine, Health Affairs, and the American Journal of Public Health. Two leading US medical journals—JAMA and the New England Journal of Medicine—published only three articles combined over the course of the 10-year period that met our inclusion criteria. We join others in calling for journals to encourage—via, for example, their core aims and scope, as well as through special issues—the submission and consideration of studies that examine social policies as structural drivers of population health and health equity (7,150-152).

This review has a few limitations. First, we included only studies that examined US policies and were published within a subset of journals and within a 10-year timeframe, to ensure the feasibility of this review. Thus, studies of policies implemented outside the US, published in other journals, and published before 2014 were not included, potentially limiting generalizability. We also narrowed our search of social policies by defining them as policies that impacted socioeconomic resources, such as income and education, while other reviews have applied a broader definition to social policies to include public health, immigration, disability, and LGBTQ+ policies (20). In addition, we further limited the policies to a list we compiled based on our knowledge of policies commonly examined in the literature, which means our search may have missed some relevant articles. Future reviews should assess the types of methods used when examining the health effects of other social policies to similarly understand trends and gaps in the literature. Finally, to maintain feasibility of the review, the two reviewers did not conduct double data extraction for the majority of articles. Instead, each reviewer extracted data from about half of the articles and only sought second opinions when necessary. Nonetheless, our use of a standardized Covidence extraction tool and piloting of it on two articles with double extraction (and 100% reviewer agreement) helped to ensure high-quality, consistent data collection.

Conclusions

This systematic review provided an overview of studies that assessed the health impacts of US social policies published recently in top health journals. We focused on methods used and tracked the consistent use of both quasi-experimental and non-quasi-experimental approaches over the study period. We identified areas for future research, including increased use of quasi-experimental designs, enhancements to data infrastructure to better capture actual exposure to policies, and more focus on policies related to housing and education. These recommendations have the potential to enhance the literature on the health effects of social policies, thereby informing policymaking to make positive impacts on population health and health equity.