Commentary: Long-Term Monitoring of Health Inequalities in Scotland—A Response to Frank and Haw

Frank and Haw (2011) devised a set of criteria that can be used to evaluate the utility of frameworks for monitoring health inequalities. They argued that a high-quality monitoring framework should ensure the completeness and accuracy of reporting, that the measures used should be reversible and sensitive to intervention, that the measure should be statistically appropriate, and that there should be no reverse causation between the proposed outcome measures and the markers of socioeconomic status. They applied these to the Scottish Government's long-term monitoring framework for health inequalities (Scottish Government 2011) to highlight the potential pitfalls for policymakers. While we welcome their description of the Scottish Government's measures as “state of the art” and recognize that there is always room for improvement, we disagree with some aspects of their appraisal of the Scottish monitoring framework, as well as the criteria they proposed. We contend that their application of these criteria to the Scottish example reveals some of the limitations of their approach.

First, Frank and Haw suggested that some of the outcome measures in the framework are not reversible or sensitive to policy change, citing as evidence the slow changes in inequalities in most of the Scottish indicators. We believe that this is an overly narrow view of the capacity of government to influence population health inequalities. Lack of change in Scotland (or any other country) is not evidence that inequalities are insensitive to policy. Moreover, it is clear that health inequalities are strongly influenced by socioeconomic policy and that change can occur rapidly (Beckfield and Krieger 2009; Mackenbach et al. 2003). Health inequalities in the United Kingdom rose rapidly during the 1980s and 1990s (Shaw et al. 2003), leaving Scotland with some populations whose mortality rates increased in absolute terms (Norman et al. 2011). Indeed, there is evidence of rather dramatic changes in inequality over time and around the world resulting from a mix of government policies (including those influencing the social determinants of health), specific health improvement interventions, and health care (Beckfield and Krieger 2009).

Second, Frank and Haw attribute the insensitivity of inequalities indicators (especially coronary heart disease, all-cause mortality, and healthy life expectancy) to their undue dependence on later-life mortality or cumulative life-course experience. It is clear, however, that inequalities in mortality in Scotland are very high among young adults (Leyland 2004; Leyland et al. 2007; Norman et al. 2011). Furthermore, mortality inequalities worldwide show marked temporal and geographical variation, suggesting that socioeconomic determinants of health have a profound impact on these outcomes (Beckfield and Krieger 2009). Macintyre reviewed the evidence for policies likely to reduce health inequalities (Macintyre 2007), and we contend that there is further scope in Scotland and elsewhere for cross-sectoral policy informed by evidence. The relative stability in recent years of inequality measures is as likely to be due to policy failure as to a lack of amenability. We therefore argue that these measures are an appropriate part of a framework aimed at “long-term” monitoring, although that does not preclude the inclusion of other, short-term measures.

Frank and Haw are similarly concerned that the Warwick-Edinburgh Mental Well-Being Scale (WEMWBS) is insensitive to change. Given that the WEMWBS is relatively new (Tennant et al. 2007), it seems somewhat unfair to dismiss it so quickly. It is equally possible that there have been no true population-level changes in this outcome over the relatively brief time period of interest, which might be expected given the trends in well-being witnessed in other rich countries (Lane 2001). However, the WEMWBS score was sensitive enough to detect changes in response to a parenting intervention (Lindsay et al. 2008), and it also shows inequalities between social groups (Scottish Government 2011). Although Frank and Haw regard these differences as small, the difference between the most and the least deprived groups is about 10 percent of the total achieved by the least deprived group. Furthermore, as with all such composite survey-derived measures, the scale is largely arbitrary. We view this measure as promising, particularly given the striking lack of suitable alternatives.

We accept Frank and Haw's concern that trends in low birth weight (LBW) reflect not only changes in the health outcome (and influences such as poverty, smoking, and nutrition) but also changes in clinical practices (e.g., inducing delivery early owing to the risk of stillbirth) and the increased survival of premature babies who previously would have been counted as neonatal deaths. A simple change here would be to report in addition on inequalities in birth weight adjusted for gestational age, with a comment on the potential for changes in clinical practice to have an influence. But it is worth noting that prematurity is also closely related to infant mortality and morbidity—and LBW is a reasonable marker of this (Smith et al. 2010).

Frank and Haw criticized the alcohol-related mortality measure in relation to its statistical appropriateness. They noted the nonlinear pattern across socioeconomic groups and suggested that segmented (spline) regression (Young 2010) may be preferable to the linear regression method conventionally used to calculate the slope index of inequality (SII). In our view, the spline approach presents several problems of its own. Departures from linearity are a matter of degree, and statistical techniques dependent on hypothesis testing (Sergeant and Firth 2006) are more likely to detect minor nonlinearity in large data sets. Spline methods also may give undue influence to outlying values in the extreme quantiles. It is not clear that an SII calculated using a spline approach is comparable between periods and areas that show varying degrees of linearity in inequalities. The important point is, of course, that all measures of inequality are imperfect summaries. We are not persuaded, either, that the alcohol-related mortality measure is so nonlinear as to make the measures invalid or that Frank and Haw's proposed alternative avoids these problems.

Frank and Haw suggested, too, that another issue in relation to statistical appropriateness is heterogeneity of outcomes; that is, some outcomes capture a mixture of conflicting trends that conceal real, but divergent, changes (Frank and Haw 2011). This is a justifiable concern in relation to the “all-cancer” indicator, since it encompasses a range of outcomes with varying determinants, preventability, and treatment. Furthermore, screening for cancers is likely to increase incidence and overdiagnosis (Gøtzsche and Nielsen 2011). Accordingly, there is merit in presenting trends in inequalities for specific cancers, but this would be dependent on having sufficient statistical power, which is clearly an issue for smaller countries like Scotland. The concern expressed about the potential impact of out-of-hospital deaths on inequalities in hospital admissions for myocardial infarction is one that we accept and that could be resolved by including these deaths.

Frank and Haw criticized the alcohol-related mortality indicator because of its susceptibility to reverse causation. While this is theoretically possible, previous reviews argued that this was a minor cause of health inequalities (Macintyre 1997). Most longitudinal studies that tested this “health selection” theory concluded that the concentration of ill health (including for alcohol-related deaths) in lower social groups is largely explained by premorbid social status rather than downward social movement (Davey Smith et al. 1998; Macintyre 1997). Furthermore, the Scottish parliament recently voted to introduce a minimum unit price for alcohol, an intervention whose impact on inequalities in alcohol-related mortality will be important to monitor.

In summary, the four broad criteria laid out by Frank and Haw seem reasonable at first sight. In practice, however, none of them has a straightforward application in the example they chose to test. Complete and accurate statistics (e.g., with individual measures of socioeconomic status) are ideal but often not available. We agree that measures should be statistically appropriate, but the statistical approach that Frank and Haw suggested seems to create as many problems as it would solve. We agree, too, that measures should be reversible and sensitive to intervention, but reversibility is not easily assessed, and a lack of change over time should not be interpreted as irreversibility. The apparent implication that the lack of reduction in health inequalities in many high-income countries in recent years indicates an inability to reduce health inequalities in the future seems unduly pessimistic. In relation to the last criterion, reverse causation is important in principle, but evidence of reverse causation needs to be carefully assessed. We have tried to make the case here that this plays only a very small part in explaining health inequalities in high-income countries. Consequently, including this criterion in high-income countries is unhelpful and unjustified in suggesting that part of the inequality in the outcome might not be unfair. Overall, Frank and Haw's critique of the Scottish Government's long-term monitoring framework is unjustified. In our view, this framework provides a valid, robust, and sensible approach to measuring progress, given the limitations of the data that are currently available or likely to be so in the near future.