Cylus et al. Respond to “Unrealized Benefits?”

We thank Dr. Bruckner (1) for his thoughtful critique of our article (2), which highlights the many challenges involved in determining how social policies affect health. Dr. Bruckner raises several interesting methodological issues. A valid point is that we cannot establish whether effects of unemployment benefit programs occur only among unemployed persons who actually receive benefits or whether benefit programs might prevent suicide among other populations who are not in receipt of benefits. In response, Bruckner suggests that we investigate effects of unemployment benefit programs in the population that is actually eligible to receive those benefits (1). We caution against this approach, for several reasons. First, changes in unemployment benefits affect not only the income of workers themselves but also that of their family members, regardless of their labor-market status. This may lead to spillover effects for persons not directly eligible for benefits and may explain why we found no significant differences in the relationship between unemployment benefit programs and suicides by age group.

Second, the eligible population is not a static group, so restricting the sample will introduce bias due to compositional changes in the eligible population over time. Importantly, these compositional changes may result from the policy itself: Some studies suggest that an increase in unemployment benefits may raise the duration or incidence of unemployment (3, 4), for example, by lowering job-search intensity among the unemployed (5). Thus, while benefits might mitigate income losses for unemployed workers, they may also increase joblessness among workers who would otherwise have stayed in their jobs or returned to work under a less generous benefit regime—thus altering the composition of the unemployed eligible population. This raises an important issue that is often overlooked: While social policies may improve the health of some segments of the population, they may also harm the health of other population groups. A “population-level” estimate captures the net effect of these or other potentially offsetting mechanisms on state suicide rates—an estimate that is at least as important to policy-makers as the effect of benefits on the eligible population.

We welcome Bruckner's proposal for a longitudinal analysis of suicide among persons who are eligible for generous unemployment benefits (1). However, given that both suicide and unemployment are rare events, we estimate that in order to have sufficient statistical power to identify an effect, a yearly sample of nearly 2 million individuals would be required. While large registry-linked data sets are available in some countries (6), we are not aware of any longitudinal data set of this size linking suicide and employment characteristics in the United States. Our approach partly overcomes this “small sample” problem by exploiting routine suicide statistics and linking them to unemployment benefit laws.

As Bruckner points out (1), our graphical results would seem to suggest that benefits have a perverse effect on suicide when unemployment rates are low (2). While interesting and plausible, we note that estimates of the association between benefits and suicide when unemployment was low were imprecise and had wide confidence intervals that always included the null value. Bruckner also raises the valid concern that changes in maximum unemployment benefits may be too small to yield meaningful effects on mental health. During our study period (1968–2008), annual changes in state real total maximum benefits averaged 0.3%, but the changes ranged from −33.4% to 51.4% (7). Large swings most often occurred when policy-makers altered the maximum number of weeks for which workers could receive benefits. The magnitude of changes in benefits was thus potentially large, at least in some cases, enabling us to identify an effect.

We believe our paper provides novel evidence that unemployment benefits may help prevent suicide during recessions, although the effects are of a small magnitude (2). A rigorous quasi-experimental study design often comes at the price of less detailed measurement of mechanisms and subgroup analyses. We chose what we thought was the most rigorous design, but we welcome future investigations focused on potential mechanisms and populations for whom unemployment benefits might offer an escape from the negative mental health outcomes associated with economic downturns.