Developing an Evidence Base for Policies and Interventions to Address Health Inequalities: The Analysis of “Public Health Regimes”

Abstract

Systematic reviews have become an important methodology in the United Kingdom by which research informs health policy, and their use now extends beyond evidence-based medicine to evidence-based public health and, particularly, health inequalities policies. This article reviews the limitations of systematic reviews as stand-alone tools for this purpose and suggests a complementary approach to make better use of the evidence. That is, systematic reviews and other sources of evidence should be incorporated into a wider analytical framework, the public health regime (defined here as the specific legislative, social, political, and economic structures that have an impact on both public health and the appropriateness and effectiveness of public health interventions adopted). At the national level this approach would facilitate analysis at all levels of the policy framework, countering the current focus on individual interventions. It could also differentiate at the international level between those policies and interventions that are effective in different contexts and are therefore potentially generalizable and those that depend on particular conditions for success.

The reduction of health inequalities has been an important part of the political agenda in the United Kingdom since the election of New Labour in 1997. An associated imperative has been that policies designed to address health inequalities should be based on reliable evidence. Drawing on experimental and quasi-experimental trials, a significant body of research has sought to differentiate influential interventions from impotent initiatives in areas relating to a wide range of health inequalities. These extend from smoking cessation and nutrition during pregnancy, breastfeeding initiation and duration, and parenting education and support, through to diet, physical activity, smoking, and alcohol consumption in adulthood and older age. According to this large evidence base, remarkably little “works” and public health researchers and policymakers are becoming concerned that this is more a commentary on the methods used in reviewing public health interventions than on the potential effectiveness of the interventions themselves (see, e.g., Kelly et al. 2005; Weightman et al. 2005).

This article begins by exploring the growing critique of the narrow, hierarchical approach to evidence in public health which, in the United Kingdom at least, usually places the systematic review and its basic building block, the randomized controlled trial (RCT), at the top of the hierarchy, whereas observational studies or expert opinions are seen as close to the bottom. We have used both our own experience of interrogating the evidence base to determine how policy and practice relate to the key sources of risk across the life course (Asthana and Halliday 2006) and a synthesis of the published literature. We found a number of limitations to systematic reviews, ranging from the quality of the primary research to the process by which the evidence is gathered, synthesized, and disseminated. These limitations include the tendency to select studies on the basis of the research design rather than the quality of the intervention; the questionable relevance of results yielded in controlled, scientific, and well-resourced conditions to the delivery of routine services; a bias toward those interventions with a medical rather than a social focus and those targeting individuals rather than communities or populations; and, central to the thesis of this article, a failure to control adequately for context, thereby undermining the extent to which the current evidence base offers lessons that are either practically applicable or transferable.

Such concerns have given rise to both a quest to improve the process of systematic review and a search for alternative strategies (Jackson and Waters 2005; Mays, Pope, and Popay 2005; Pawson et al. 2005). Such strategies advocate methodological pluralism, with most of the leading health inequalities researchers also emphasizing a need to shift from the current focus on downstream policies (i.e., policies focusing on specific risk factors at the individual level, such as smoking cessation) to upstream policies targeting the wider determinants of health, such as income distribution, employment, education, access to important services, and laws and regulations pertaining to health-damaging exposures (Davey Smith et al. 1999; Davey Smith, Ebrahim, and Frankel 2001; Hertzman 1999; Independent Inquiry into Inequalities in Health 1998; Mackenbach and Stronks 2002; Townsend 1999). However, the current approach to gathering and reviewing evidence is ill-equipped to assess the effects of different policy and organizational arrangements in these broad domains.

Natural policy experiments (where observational data on a treatment group and a control group exist without deliberate randomization) that draw on evidence from other countries offer a potentially promising approach to gathering evidence on upstream policies. But to date, the focus of international comparative work on policies to reduce health inequalities has leaned toward country-specific descriptive case studies or decontextualized systematic reviews, neither of which yield firm conclusions about the factors that might influence observed differences in policy approaches or outcomes. To improve the utility of natural policy experiments, we believe that comparative research on policies to reduce health inequalities and interventions should, at the very least, involve a systematic assessment of the structures and institutional arrangements that both lead to health variations and shape the formation, implementation, and effectiveness of policies and interventions designed to reduce health variations.

Following a critique of the current evidence base, the second part of this article offers a new research agenda to inform policies targeting health inequalities. It uses methods developed within comparative social policy, involving the analysis of public health regimes, to propose a framework capable of facilitating both contextualization and comparison. This approach, we argue, would have real-world relevance. It would help address the lack of systematic evidence regarding the role and impact of upstream initiatives in reducing socioeconomic inequalities in health. It also would focus attention on the significance of contextual features in determining the impact of downstream initiatives. Both features are shortfalls that our critique suggests constrain the utility of the current evidence base. Perhaps most significantly, however, this approach would enable policymakers and practitioners, first, to unpack the complex concept of context into its component parts and, second, to understand better the consequences of working in different contexts (McCormack et al. 2002). It would thus help identify those policies and interventions that are effective in a range of contexts (and therefore potentially generalizable) and those that depend on particular conditions for success.

Evidence-Based Policymaking and Systematic Reviews

Evidence-based medicine came to prominence in the United Kingdom in the early 1990s, offering an alternative and more accountable basis to clinical expertise for decision making in the field of health (Elliott and Popay 2000). It was initially applied to clinical practice, but the New Labour government soon introduced a more pervasive requirement for policymakers to use “evidence and research so that we understand better the problems we are trying to address” (Cabinet Office 1999a, 17), and the approach was extended to public health (defined to include both the policymaking process and the activities of public health agencies) and health improvement (Kelly, Speller, and Meyrick 2004).

This extension was expected to be comprehensive and to include the use of the best available evidence from a variety of sources, the systematic evaluation of the effectiveness of policy, an ongoing review (so that redundant or failing policies would be scrapped), and the ability to learn lessons. The process was accompanied by the establishment of new government units (such as the Performance and Innovation Unit, the Social Exclusion Unit, and the Centre for Management and Policy Studies within the Cabinet Office) to help public servants acquire the skills and tools needed to reform and modernize public services (Bullock, Mountford, and Stanley 2001; Cabinet Office 1999b). It also was accompanied by an enlargement of the institutional base supporting the production of evidence. For example, the Campbell Collaboration (established in 1999) extended the Cochrane Collaboration (whose major focus is on clinical treatments) to the social, behavioral, and educational fields, where it places a similar emphasis on the preparation, maintenance, and dissemination of systematic reviews of studies of interventions. The Evidence for Policy and Practice Information and Co-ordinating (EPPI) Centre (funded by several government departments and charities) addresses the need for a systematic approach to the organization and review of evidence-based work on social interventions, including promoting Cochrane Reviews of health promotion and public health topics. The National Institute for Clinical Excellence (NICE), now the National Institute for Health and Clinical Excellence, was established to provide national guidance on the promotion of good health and the prevention and treatment of ill health. The Social Care Institute for Excellence (SCIE) was formed to support the quality and consistency of social care practice (a term covering the services provided by local authorities and independent and voluntary sectors to support vulnerable adults and children), including the views, experiences, and expertise of users and caregivers. Finally, the ESRC (Economic and Social Research Council) UK Centre for Evidence Based Policy and Practice promotes the concept of evidence-based policy and practice in the social sciences.

The UK government itself has been receptive to methodological pluralism in the collection of evidence. In practice, however, it is the systematic review and the RCT that have come to occupy a central position in the construction of the evidence base for policies to reduce health inequalities. As we explain later, this situation has provoked debate about the ability to transfer the methods developed for the field of evidence-based medicine to the more complex task of synthesizing evidence for public health policy (Petticrew et al. 2004). Concerns have also been expressed about the tenuous relation between the evidence base, however defined, and the process of practical policymaking, which continues to be “about ‘muddling through’ rather than a process on which the social or policy sciences have had an influential part to play” (Parsons 2002, 43).

The Advantages of and Drawbacks to Systematic Reviews

The advantages of systematic reviews are inherent in their definition. They are carried out according to agreed standards (including the focus on a specific question); attempt to identify as much relevant research as possible; consider studies for appraisal and synthesis only if they meet defined quality standards (e.g., involve randomized control); and use protocols to guide the process and facilitate replication (Boaz, Ashby, and Young 2002). In the process, systematic reviews aim to reduce bias and synthesize research findings in a way that is accessible to those who have to make policy or practice decisions. The two best-known techniques used in systematic reviews are meta-analysis and narrative synthesis. Meta-analysis uses statistical techniques to bring together the findings of the studies that meet the inclusion criteria and provides an arithmetic summary of net success. Narrative synthesis looks at the similarities and differences between the studies and their outcomes without calculating an average effect size and makes recommendations not by quantification but by the use of exemplars. The diversity of interventions to be considered in health inequalities policy and the multiplicity of outcomes tend to discourage numerical abstraction. Most systematic reviews in this field, therefore, have taken the form of narrative review rather than meta-analysis.

The drawbacks to extending systematic reviews to wider public policy interventions, including those relating to health inequalities, can be divided into two broad classes: those that address the hierarchy and logic of the review process and those that address the quality of the primary research.

The Hierarchy and Logic of the Review Process

Three drawbacks to systematic reviews pertain specifically to the methodology. The first objection is that the methodology underplays the role and the development of theory, emphasizing those interventions that produce statistically significant outcomes rather than emphasizing the process, that is, how such interventions achieve this effect, the circumstances in which they are effective, and the population groups for which they work. Two important reservations follow. The first is that “without the theory of what is important we are stuck with trying to duplicate the whole thing” (Pawson and Tilley 1997, 191; also see Haynes 2003). The second is that with so many possible variables relating to every intervention, the outcomes are often not statistically significant, so they leave the policymaker with an array of inconsistent results rather than a better understanding of the options available.

As interventions move from the medical to the social domain, their complexity intensifies, and such problems become more pronounced. For example, Towner and colleagues' (2001) narrative review of interventions to prevent accidental injuries to children and adolescents extends from individual and behavioral approaches (such as safety equipment and road safety education) to environmental modifications (such as calming traffic and legislation). Other variations relate to the age of the children concerned, the nature of the environment (from roads to sports), the approach taken, and whether the intervention addresses one issue only or is multimodal. In such circumstances, it might be more logical to test the same policy idea in different settings rather than to compare variants working through very different program mechanisms. It is noteworthy that while RCTs were widely used in the United States throughout the 1960s and 1970s in areas relevant to health inequalities, such as income maintenance, supported work environments, housing allowances, and support for prisoners, they have since become less extensive in the face of limited statistical evidence as to the efficacy of such interventions (Oakley 1998).

A second fundamental objection is that the studies to be included in systematic reviews are selected on the basis of the research design (defined according to preordained criteria), not the quality of the intervention (e.g., its theoretical basis, relevance to the target population, adequacy of resources, or duration) (Davey Smith, Ebrahim, and Frankel 2001; Speller, Learmonth, and Harrison 1997). A hierarchy of evidence that favors the systematic review because of its assumed methodological strengths (chiefly its reliance on RCTs) therefore may “attenuate public health decisions,” preferring “interventions with a medical rather than a social focus, those that target individuals rather than communities or populations, and those that focus on the influence of proximal rather than distal determinants of health” (Rychetnik et al. 2002, 125). This does not necessarily mean that interventions aimed at whole communities are not effective but instead reflects “the paucity of good quality studies of these more ‘upstream’ interventions” (Macintyre et al. 2001, 224). A third, closely related, reservation is the dissonance between systematic reviews, which focus largely on individual interventions, and health inequalities, which are not primarily generated by medical causes and so require solutions at a different level, such as the redistributive effects of national fiscal policies, or economic investment to counter unemployment (Davey Smith, Ebrahim, and Frankel 2001, 185).

The Quality of Evidence

The second drawback is the quality and scope of the primary research, which limits the ability of systematic reviews to yield clear findings. Methodologically sound studies for systematic review in the field of health inequalities have been found to be “disappointingly scarce” (Harden, Weston, and Oakley 1999, cited in Boaz, Ashby, and Young 2002). These studies have a number of important limitations. First, even the included studies often fail to report on the content, duration, and intensity of the intervention, making it difficult to discriminate between evaluation failure and program failure or between the contributions of different program elements. For example, we know that parenting programs can make a significant contribution to the short-term psychosocial health of mothers, but we still do not know which factors contribute to the successful outcomes (Barlow and Coren 2003). Similarly, none of the reviews of home-visiting programs have been able to determine which components help reduce childhood injury in the home (Lucas 2003) despite the fact that home visits are far from being a single or uniform intervention (Bull et al. 2004, 1).

This lack of discrimination is compounded by poor or inconsistent definitions of terms. Control groups, for example, are normally described as receiving the “usual care,” without explaining what that is. Yet this care undoubtedly varies from country to country and may be different at the time of the trial from what it is now. Similar reservations apply to many other areas. The term breast-feeding, for instance, is used without commonly accepted definitions for its initiation or continuation and without distinction between partial or exclusive feeding. Likewise, smoking is often treated as a dichotomous variable, without acknowledging the very different experiences of heavy and light smokers (Amir and Donath 2002). There also are problems relating to measurement. For example, self-reported measures of diet and physical activity often are weak estimates of actual behavior and prone to distortion when subject to long periods of recall, while the contribution of smoking cessation studies that do not validate smoking status has been questioned (Lumley, Oliver, and Waters 2001).

The neglect of confounding variables such as gender, ethnicity, family composition, age, socioeconomic status, and education is another problem. Systematic reviews often have very little to say about inequalities or inequities (Petticrew et al. 2004) or about the ways in which different segments of the population respond to similar interventions. In part, this is because the most rigorous evidence is often gathered from “simple interventions and from groups that are easy to reach in a population” (Rychetnik et al. 2002, 125) rather than from interventions targeted at disadvantaged groups. An overview of research relating to the principal public health areas outlined in the UK White Paper Saving Lives and The NHS Plan, which covered intervention research, reviews, and exploratory/informative articles, found research focusing on inequalities to be limited, with no more than 0.4 percent of the output examined relating to public health intervention research. Such restrictions were particularly true of the reviews of the larger organizations like the Cochrane Collaboration and were attributed to the complexity of the issues involved, the methodology (with the public health community reluctant to settle for methodological pluralism), timescale/structure (with long-term evidence and hence long-term health gains often neglected), and theory (Millward, Kelly, and Nutbeam 2003, 31). A similar deficit was identified by the Wanless Report on the future resource requirements of the NHS, which found that more research across the United Kingdom and better data on inequalities (both socioeconomic and geographical) and their impact on health need and costs were required (Wanless 2002, 126). Instead, much of the evidence available is located far down the causal chain, concentrating on downstream proposals to address health behaviors and clinical issues rather than the broader social determinants of health.

Such methodological limitations and the neglect of key issues are further compounded by technical problems such as randomization, low participation, high rates of attrition, and an uneven proportion lost to follow-up between control and intervention groups. Significantly, attrition and recruitment also display a socioeconomic bias. For instance, attrition from parenting programs is related to poverty, high levels of stress, high levels of child conduct disorder, and ethnic minority status. Such challenges mean that it is important for the studies to analyze all subjects, regardless of whether they have received or completed the intervention, but it often is not clear whether such an analysis was undertaken (Barlow and Parsons 2003). Furthermore, the existence of ethical problems relating to the ability to give informed consent or to possible side effects prevents randomization and blinding in many public health–related interventions, such as randomizing mothers' feeding choices (Nicoll and Williams 2002). In turn, practical concerns mean that many randomized control trials take place in hospitals or clinics. Questions thus remain regarding how to translate the results obtained under controlled, scientific, and well-resourced conditions into the delivery of routine services (Kelly 2004). The evidence suggests, for example, that smoking cessation interventions in pregnancy are often less successful in real-life settings than are those conducted in clinical trials (Bull, Mulvihull, and Quigley 2003).

Further questions have been raised about the effectiveness of the quality assurance mechanisms for the primary raw material of systematic reviews, that is, papers published in peer-reviewed journals. Peer review has been seen as a guarantee of research quality (Grayson 2002), but journal conventions may distort the review process by excluding accounts of program failure, minimizing contextual information, and concentrating on positive outcomes. In addition, the focus of journals is overwhelmingly on publication in English, thereby discouraging international comparisons, although a commonality of language also can lead to inappropriate extrapolation. For example, many of the systematic reviews used to aid policy and practice in the United Kingdom are dominated by studies conducted in North America without considering the very different political and institutional contexts. This problem has been exacerbated with the move to health and social policy because a smaller proportion of potentially relevant studies appear in peer-reviewed sources than do those in medical research (Grayson 2002). Health and social policy also has a more diverse literature, a greater variety and variability of secondary bibliographical tools, more material available on the Internet, and a less precise terminology (Grayson and Gomersall 2003).

Such limitations are widely acknowledged, even by the authors of systematic reviews themselves, who deplore the paucity of primary research and the neglect of health inequalities. In this way, systematic reviews are increasingly being used to illuminate problems and raise areas of concern rather than to provide definitive answers. For example, reviews of smoking cessation programs have shown that attrition is highest for both low-income and the most mobile people (Lumley, Oliver, and Waters 2001). Similarly, a recent systematic review of interventions to prevent obesity in children indicated that “the mismatch between the prevalence and significance of the condition and the knowledge base from which to inform preventative activity” was “remarkable and an outstanding feature of this review” (Campbell et al. 2002, 12).

Some of these limitations are now being addressed in order to make systematic reviews more useful to public health policymakers (Lavis et al. 2005). New approaches are less dependent on experimental and quasi-experimental designs (Greenhalgh et al. 2004; Lynch et al. 2004), and researchers have begun to develop new tools to appraise qualitative research identified for inclusion in reviews (Harden, Weston, and Oakley 1999; Mays, Pope, and Popay 2005; Popay, Rogers, and Williams 1998; Spencer et al. 2003) and to incorporate corroborative evidence (from observational and qualitative studies) to indicate the feasibility and likelihood of an intervention's success in a particular context (Weightman et al. 2005). Other researchers have successfully used random allocation in social interventions, ranging from the effects of early-years day care centers and peer-led sexual health education to strategies for maternal social support, to demonstrate that randomization can indeed be acceptable in terms of science, ethics, and feasibility if the intervention is seen as addressing an important policy issue with a strong rationale for random allocation (Oakley et al. 2003). More emphasis also is now being placed on causal factors rather than the typical “black box” of epidemiology “that placed more weight on research methods and outcomes than on intervention theory” (Rychetnik et al. 2002, 122). Our proposed framework attempts to build on this recognized need to synthesize qualitative and quantitative research, to make use of theory, and to consider the complexity of governance, delivery, and policy problems in the field of health inequalities.

Complementary Approaches to an Evidence Base for Health Inequalities Policies

As part of a wider research project (Asthana and Halliday 2006) designed to identify interventions that effectively address the pathways producing health inequalities during the life course, we consulted more than 125 systematic reviews drawn from review and research organizations, government departments, and peer-reviewed journals and comprising at least 1,800 experimental and quasi-experimental trials. We found that the current evidence base is stronger for interventions implemented during adulthood than earlier and particularly favors those interventions targeting individual lifestyle factors (diet, physical activity, and smoking). By contrast, the evidence base for interventions addressing the wider determinants of health (e.g., housing, crime, and employment) is very weak. As a whole, the evidence base lacked information about the features (design and contextual) that might make an intervention appropriate to particular vulnerable groups. Overall, we found little evidence for effective intervention, and our task of providing a guide to “what works in tackling health inequalities” risked becoming an academic exercise in circumspection. This was perhaps inevitable given the impossibly stringent criteria for success, although this is a dangerous conclusion, as it may legitimize political inertia. It also is misleading, as some interventions work for certain groups, under certain conditions, and in certain contexts but not in others—that is, they are “conditionally” successful (Pawson and Tilley 1997).

This provided the impetus for our search for complementary strategies, particularly those whose focus was on policies relating to services and governance and where context was considered. For example, Rychetnik and colleagues (2004) recommend moving from the current emphasis on type 1 evidence that identifies the relationship between risk and disease and type 2 evidence that identifies the relative effectiveness of specific interventions (Brownson, Burney, and Land 1999) to type 3 evidence. Type 3 evidence contains more descriptive and/or qualitative information about the design and implementation of an intervention, the contextual circumstances in which it was implemented, and how it was received. The need to develop a “mixed economy” of evidence, in which policy relevance is more important than an obsession with methodological hierarchy, was similarly highlighted in a recent study of senior policymakers' perceptions of current research (Petticrew et al. 2004) and in a companion study of the views of current research leaders in the health inequalities field (Whitehead et al. 2004). These studies stressed the value of qualitative evidence (particularly when paired with quantitative data), historical evidence, and natural policy experiments and suggested that the creative assembly of an evidence “jigsaw” from such diverse sources (including observational studies and controlled evaluations of interventions) would yield far more information that was relevant to policy (Whitehead et al. 2004).

Although some scholars might assume that such studies are merely lowering the bar in regard to admissible research in order to offer policy advice, we believe that they not only retain a strong empirical underpinning but also correct one of the principal failings of the current evidence-based assessments that we discussed earlier: their focus on individual interventions. In order to fulfill the criteria of medical research, randomized control trials and systematic reviews tend to focus on downstream policies or discrete interventions, whereas public health interventions should include “policies of governments and non-governmental organisations; laws and regulations; organisational development; community development; education of individuals and communities; engineering and technical developments; service development and delivery; and communication” (Rychetnik et al. 2004, 540). We now turn to the potential role of international comparative research in providing such information.

International Comparative Research on Health Inequalities Policies

Insofar as they allow for the analysis of different legislative, social, political, and economic structures (e.g., national fiscal policies, labor market policies, legislation regarding harmful substances, and food subsidies), international comparative studies are a promising approach to gathering evidence about upstream policies. As we observed earlier, however, the evidence that does exist on health inequalities interventions tends not only to focus on downstream initiatives but also to be biased toward studies published in English and especially those from the United States. Against this background, the aims of the European Network on Interventions and Policies to Reduce Inequalities in Health are significant. Members of this interdisciplinary group prepared structured descriptions of policies that explicitly addressed socioeconomic inequalities in health in their own country between 1990 and 2001 and searched for empirical evidence of the effectiveness of specific policies and interventions to reduce these health inequalities (Mackenbach and Bakker 2003).

Rather than yielding comparative data across a range of indicators or leading to conclusions about factors that might have influenced observed differences in policy approaches or outcomes, this approach mainly provided various country-specific case studies. As the Network's coordinators observed, however, differences in policy content and context made direct comparisons difficult. For example, the countries had a different awareness of and willingness to take action on inequalities in health. The extent to which country-specific initiatives were replicable on grounds of practical and political feasibility also varied (Mackenbach and Bakker 2003).

Although these problems of comparison are legitimate, the difficulty of making cross-national comparisons is a reflection of the research design as well. The Network did not set out to analyze policies within an explicitly comparative framework. Moreover, its use of formal databases in the search strategy for literature on effective interventions also led to the exclusion of unpublished reports and studies published in national journals and to an overwhelming reference to the types of research (e.g., randomized control trials and controlled experiments) that have been criticized for their focus on individual interventions and their failure to acknowledge the importance of context. Thus, when the Network undertook cross-national work on specific domains, this mainly resulted in systematic but largely de-contextualized policy reviews (e.g., Hogstedt and Lundberg 2002; Mielck, Graham, and Bremburg 2002; Paterson and Judge 2002; Platt et al. 2002; and, with the exception of their case studies of food and nutrition policies in Finland and the Netherlands, Prättälä et al. 2002). One exception was a descriptive review of income maintenance policies which, drawing on the seminal work of Esping-Andersen (1990), highlighted important differences in policies associated with various welfare regimes (Diderichsen 2002). While this study emphasized important contextual features, it did not yield clear conclusions about the implications of different welfare systems for the prevention of health inequalities.

Although the Network could undertake only a limited comparative analysis, its descriptions of national experiences did allow its coordinators to identify examples of innovative approaches to reducing health inequalities that were supported by some empirical evidence of effectiveness. For example, the Netherlands' use of national targets relating to intermediate outcomes in the areas of poverty, labor participation of chronically ill people, smoking, and heavy physical labor was considered to have been effective. Compared with England, Sweden's strong employment protection and active labor market policies for chronically ill citizens were identified as effective ways of protecting vulnerable groups from being excluded from the labor market. France's mandatory occupational health services were seen as an effective setting for promoting preventive activities, including smoking cessation, and developments in Finland relating to health-related behaviors such as food consumption and the provision of free or subsidized meals at school and in the workplace were highlighted. Universal access to health care was considered to be an important strategy for addressing social disadvantage. Finally, the use of territorial approaches or area-based initiatives (such as the English Health Action Zones) was found to be an innovative and possibly effective approach (Mackenbach and Bakker 2003).

These examples are not necessarily an “evidence-based” list of effective upstream initiatives. The Network itself acknowledged that its evidence of effectiveness might not always meet the highest scientific standards. Moreover, interpreting the experiences of other countries in terms of the country with which we are most familiar is a recognized pitfall of comparative research on social policy (Cochrane, Clarke, and Gewirtz 2001). Indeed, many English commentators would dispute the claim that Health Action Zones have made a significant difference to the health of disadvantaged populations, and even though the UK government's strategy for reducing health inequalities acknowledges the role of a comprehensive range of social policies, the way in which that strategy has been implemented does not necessarily support the Network's conclusions (Mackenbach and Bakker 2002, 341) that Britain is at a more advanced stage than its European counterparts with regard to coordinating health inequalities policies (Asthana and Halliday 2006).

Toward a New Framework: The Public Health Regime

For public health strategies to be effective at a national level, context should be a central focus of interest. The Network's experience suggests that the utility of natural policy experiments can be improved in a number of ways. First, single-country case studies would be strengthened by a systematic assessment of a common set of structures and institutional arrangements, identified by a theory-based approach. Most researchers accept that although many health-damaging exposures and behaviors can be identified at an individual level, these are embedded within a wider socioeconomic context. Policy responses to health inequalities should therefore target wider determinants of health as well (i.e., through the development of upstream policies). What is less often acknowledged is that structural factors (relating to society, economy, culture, and policy) also shape the formation, implementation, and effectiveness of downstream interventions.

Given the progress in understanding the pathways that lead to health inequalities in different stages of life, well-designed single-country studies should be amenable to an assessment of the potential policy's effectiveness by using a theory-based approach, for example, by “surfacing” the implicit theory that links intervention activities with desired outcomes in the short, medium, and long terms (Connell and Kubisch 1998). But to find examples of policies that work regardless of context or to clarify the conditions necessary to transfer effective policies from one context to another, we need both contextualization and comparison.

An Analytical Framework for Public Health Regimes

The aim of our proposed framework for international comparison (table 1) is not to make methods like the systematic review obsolete but to incorporate them into a broader approach that, with its attention to context and process and its ability to accommodate upstream and downstream interventions, could be more helpful to policymakers. This framework has been influenced by such sources as realistic evaluation, where the stress is on the relationship between context, mechanisms, and outcomes (Pawson and Tilley 1997); complexity theory, with its interest in associations and interactions among factors (Sanderson 2002); and our own research (Asthana and Halliday 2006) on the pathways that lead to health inequalities and the way in which interventions designed to address such pathways are socially embedded. Our framework has five central features. The first focuses on the broad domains of public health regimes—their political, legal, social, cultural, economic, and organizational structures—and covers the wider contextual factors that affect inputs, processes, and outcomes relating to the reduction of socioeconomic inequalities in health. With the exception of the cultural domain, each can be linked to a core set of beliefs and values along a spectrum ranging from a belief in individual self-determination and the role of the market, a tolerance of inequality and an antipathy to government intervention, to a belief in equality, social justice and solidarity, and a strong role for the state.

TABLE 1.

Public Health Regimes: An Analytical Framework

		Political Ideology
Domain	Sample Indicators (Using Nutritional Inequalities as an Example)	Political Right		Political Left
Political:
Welfare regime	Qualitative description	Liberal	Corporatist	Social Democratic
	Gross public social expenditure relative to GDP	<20%		>40%
Role of state in welfare funding and provision	Net public social expenditure relative to GDP (i.e., adjusting for tax and benefit systems)	<20%		>30%
	Decommodification score	<15		>35
Access to welfare services	Access to free school meals, employment-based catering, nutrition programs for MCH	Safety net	Subsidized	Universal provision
Core political values	Qualitative descriptions of core political values	Markets, individualism, choice		Equality, social justice, solidarity
Legal:
Strength of legislation	Nature of controls on food advertising, food standards, etc.	Minimum	Voluntary	Compulsory
Social:
Social structure	Income distribution	High inequality		Low inequality
	Relative poverty rates	High		Low
	Educational performance and inequalities	High disadvantage		Low disadvantage
	Qualitative description of key social divisions
Economic:
Level of marketization/ consumer orientation	Food price, subsidy, and tax policy measures	Weak		Strong
	Degree of trade liberalization	Tendency toward liberalization		Tendency toward protectionism
Relative power of statutory and commercial interests	Ratio of statutory and commercial funds spent on food promotion	Low		High
Organizational:
Degree of policy development and coordination	Mechanisms for food policy coordination (e.g., national/local food policy councils)	Weak		Strong
	Focus of key task forces (including biological/environmental, ethical)	Narrow		Holistic
	Range of stakeholders involved in policy coordination and implementation	Narrow		Wide
Cultural:
Sociocultural meanings of food	Qualitative analysis
Traditional and contemporary food patterns	Data from surveys and surveillance systems
Extent of globalization regarding food preferences	Analysis of food imports, etc.

A second feature of our framework is its use of indicators to construct a summary of each domain in a country's public health regime. Providing that they can be compared, these indicators may be qualitative or quantitative, but the choice of indicator should be based on theory (i.e., the indicators are linked to either causal factors that lead to health inequalities or contextual factors that shape the interventions' effectiveness). In practice, this demand for deriving theory-based indicators suggests that (as with systematic reviews) this approach will be easier to apply to interventions targeting specific risk factors (e.g., smoking, nutrition) or preventable diseases and conditions than to a general examination of health inequalities. Thus, a third feature of the framework is the extent to which it can be tailored to different uses, a characteristic reflected in Table 1, which illustrates the framework with respect to the analysis of policies and interventions targeting nutritional inequalities.

Although indicators are used to depict relative position on the spectrum of core political principles, we do not assume that ideological orientation and public health outcomes are automatically correlated. Instead, we use domains and composite indicators to provide a contextual framework for single-country studies of key structures, institutions, and outcomes relating to public health or to aid international comparative analysis of specific interventions. For single-country studies, our proposed framework allows a comprehensive analysis of interventions at all levels of the policy framework (i.e., upstream and downstream), which can be assessed on a theoretical basis using existing knowledge of the pathways that lead to health inequalities. More ambitiously, international comparative analysis provides a vehicle with which to identify regularities and differences in the outcomes of public health interventions and to ask questions about the role and impact of different structural factors in reducing health inequalities. For example, what are the main differences and similarities in national structural, policy, and organizational arrangements pertaining to public health? Are some regime types and domains more conducive than others to the promotion of health equity? Is it possible to distinguish between those interventions that are effective across different public health regimes (and therefore are generalizable) and those that are successful only under certain—identifiable and possibly replicable—conditions and contexts? Can local action address health inequalities without the assistance of enabling structures at the national level? The fourth feature of our framework is therefore that it can be used to generate series of hypotheses about contexts, mechanisms, and outcomes regarding the reduction of health inequalities.

Through the generation of hypotheses subjected to empirical evaluation, we believe that an analysis of public health regimes could enhance our understanding of the relative role of different domains in determining the effectiveness of policies and interventions to address health inequalities. This, the fifth feature of our framework, could provide a basis on which to apply weightings to different indicators and domains (e.g., expressed by different line widths when marking their position along the spectrum of core values). Such an approach could usefully reveal the complex and often ambiguous status of different countries' welfare and public health regimes. In the United Kingdom, for example, much policy attention has been paid to the problem of health inequalities, and the government has explicitly recognized the links between the underlying inequalities in income, wealth, and poverty and inequalities in health. Yet neoliberalism is apparent as well in the UK government's support for the wider economic forces that favor socioeconomic polarization, leading some commentators to suggest that the government's progress on eradicating health inequalities has fallen far short of its rhetoric (Asthana and Halliday 2006; Shaw, Davey Smith, and Dorling 2005).

Implementing the Framework

As our critique indicates, systematic reviews have not been amenable to the evaluation of the impact of upstream policies on health inequalities, a failing that reflects the tendency of epidemiological research and evidence-based medicine to conceptualize health risk factors and related interventions at an individual level. Yet as Graham (2004) suggests, the mechanisms that generate and distribute power and wealth are the same as those that stratify health outcomes. One purpose of our proposed framework is thus to explicitly surface the wider social determinants of variations in health and show which interventions will likely be the most effective in different contexts.

By incorporating contextual variables and embracing a mixed economy of evidence (including information on the process by which interventions are implemented), a second purpose of our framework is to improve for policymakers the utility of research such as systematic reviews of intervention effectiveness. The fact that some interventions work in certain contexts but not in others suggests the need to test policy ideas in different settings. Thus, researchers reviewing intervention studies and their outcomes should, when possible, not only treat context as a key variable but also consider and categorize its component parts. This requirement is clearly acknowledged in the concept of the public health regime. In the theoretical schema, contextual variables relate primarily to the national level. In practice, however, the framework could be readily adapted to include local contextual variables (e.g., relating to organizational, community, and institutional conditions).

As we noted earlier, there are encouraging signs that new approaches to systematic reviews are receptive to the difficulties of applying traditional criteria to public health interventions. One response has been to expand search strategies to include qualitative as well as quantitative material, and unpublished reports and studies published in national journals as well as research published in international peer-reviewed journals. Because of the tendency to conflate high-quality evidence with randomization and the identification of population-level effects, the use of more diverse sources (including local evaluations of targeted initiatives) could start to remedy the paucity of studies on disadvantaged groups and better capture issues relating to the processes of implementation. The wealth of international comparative data (descriptive and statistical) could be better exploited and would be amenable to analyses of the relationship between different public health arrangements and important public health outcomes. A substantive body of research, for example, has already examined the relationship between income inequality, life expectancy, and mortality in order to establish whether aggregate income inequality has a direct health effect over and above that produced by individual income (e.g., Lynch et al. 2004; Mellor and Milyo 2001; Subramanian, Blakely, and Kawachi 2003; Wilkinson 1992). While it is important to recognize that different aspects of the public health infrastructure are empirically difficult to disentangle (Lynch et al. 2004, 12), this approach could be usefully extended to examine associations between other aspects of social policy and other health outcomes.

Conclusion

Public health interventions to address health inequalities, however well designed, operate within complex social systems. Not all the variables that could affect the implementation and impact of an intervention can be controlled. After acknowledging that the effectiveness of similar interventions can vary according to context, the main task instead is to distinguish between potentially generalizable and conditionally successful interventions and, for the latter, to find those contextual features that turn potential into successful outcomes. We believe that the international comparison of policies and interventions adopted by different public health regimes can be a useful analytical framework to this end. The next task is to analyze policies and interventions more systematically within this framework. Accordingly, systematic reviews will continue to be a helpful tool—provided that they are used in this broader framework together with other sources of evidence. We believe that an exciting research agenda can be developed and implemented in the field of evidence-based public health. A key question is whether researchers and policymakers are ready to accept the limitations of the current hierarchical approach to informing policy and practice and to consider the concept of public health regimes more closely when deciding what evidence is most useful to their specific situation. Recent guidelines are encouraging, as they argue for reviews that meet the needs of users, acknowledge the significance of context, and admit a plurality of evidence (Jackson and Waters 2005).