Courtin and Muennig Respond to “Trials of Social Determinants”

We thank Rojas-Saunero et al. for their commentary (1) on our article, which assessed the mental health effects of expanding the Earned Income Tax Credit for low-income Americans without dependent children in a randomized controlled trial (RCT) (2). We agree that evaluating the health effects of experimental social interventions presents challenges that extend beyond those of, for example, drug trials.

As Rojas-Saunero et al. point out, follow-up was 69% at 3 years into our study (2), so we do not know how the other 31% of participants would have responded to the intervention. Provided that there is no differential attrition, it is not implausible to assume that treatment effects were observed at random. Linkages to administrative data can both fill in these gaps for trials in progress and be used to assess the long-term impacts of trials conducted in the past (3). The United States’ unique history of conducting social policy RCTs since the 1960s is actually a treasure trove for social epidemiologists interested in assessing the long-term effects of social policy interventions, and administrative data offer superior follow-up to most clinical trials. Mortality data are one example of administrative data that can be linked to older trials, providing of course that the original identifiers can be found and that the consent process allows for the linkage.

Rojas-Saunero et al. also point to problems that pose even more serious challenges to study validity. For example, they point out that social policy experiments are usually not blinded. In practice, it common for participants to be informed of the treatment and control conditions. This allows them to guess their treatment assignment.

We also agree with the authors on the importance of thinking of alternative protocols and joint interventions from the start of the interventions. In practice, this will prove challenging. Health researchers often piggy-back on existing RCTs, which are conducted for the purpose of assessing the socioeconomic impact of the intervention (4). Historically, health outcomes have been an afterthought in most of the social experimentation literature rather than the measure of interest (4). As national health budgets balloon, there are growing calls to consider health effects of social policy interventions in the cost-effectiveness evaluation of costly welfare policies (5). This field is in its infancy, and many more interdisciplinary studies need to be conducted and planned with health experts for the ambitious program outlined by Rojas-Saunero et al. (1) to come to life.

Experimental studies of social policies are certainly no panacea. However, they do play a critical role in providing policy-makers with additional causal data that compliment observational and quasi-experimental studies. It cannot be overemphasized that, until recently, there was considerable debate surrounding whether increased income can improve health, let alone whether it is possible to improve the health of low-income adults with a social policy intervention (6). Some of the suggestions made by Rojas-Saunero et al. can be implemented immediately, and we will certainly do so ourselves in ongoing and future work. It is important, though, not to make the perfect the enemy of the good, because it is not possible to make informed policy decisions without reasonable certainty that large taxpayer investments will produce health and economic benefits.