When is a randomised controlled trial health equity relevant? Development and validation of a conceptual framework

Abstract

Background

Randomised controlled trials can provide evidence relevant to assessing the equity impact of an intervention, but such information is often poorly reported. We describe a conceptual framework to identify health equity-relevant randomised trials with the aim of improving the design and reporting of such trials.

Methods

An interdisciplinary and international research team engaged in an iterative consensus building process to develop and refine the conceptual framework via face-to-face meetings, teleconferences and email correspondence, including findings from a validation exercise whereby two independent reviewers used the emerging framework to classify a sample of randomised trials.

Results

A randomised trial can usefully be classified as ‘health equity relevant’ if it assesses the effects of an intervention on the health or its determinants of either individuals or a population who experience ill health due to disadvantage defined across one or more social determinants of health. Health equity-relevant randomised trials can either exclusively focus on a single population or collect data potentially useful for assessing differential effects of the intervention across multiple populations experiencing different levels or types of social disadvantage. Trials that are not classified as ‘health equity relevant’ may nevertheless provide information that is indirectly relevant to assessing equity impact, including information about individual level variation unrelated to social disadvantage and potentially useful in secondary modelling studies.

Conclusion

The conceptual framework may be used to design and report randomised trials. The framework could also be used for other study designs to contribute to the evidence base for improved health equity.

Strengths and limitations of this study.

• Randomised controlled trials (’randomised trials’) can provide evidence on both the impact of interventions in general and differential impacts in different populations. This is critical for decision-making to address inequity.

• A consensus building process was used to develop and validate a conceptual framework to identify and classify health equity-relevant randomised trials.

• The proposed conceptual framework engages users in a deliberative process to design and report studies to provide evidence that is useful to understand the distribution of effects rather than only average effects.

• A limitation is that the analysis and publishing of disaggregated data can be expensive and may not be desirable for all randomised trials.

• The conceptual framework may be used to design health equity-relevant randomised trials (as well as other study types) and to identify health equity-relevant studies that contribute to an evidence base that improves overall health and health equity.

Introduction

Many countries have policy documents that reflect concern for social justice, with a goal to redressing disparities in health both within and between countries.¹ For example, within Canada, the difference in life expectancy between Indigenous and general populations is on average 10 years.² Differences in health between individuals or population groups within society, or health inequalities, are labelled ‘inequities’ when considered unfair.³ There are differing views on what counts as unfair and moral judgements about fairness may vary over time and between cultures.⁴ For example, men have a shorter life expectancy (with less disability) than women, and there is controversy about whether this is inequitable.⁵ Two common guiding principles, frequently advocated in public health policy documents, are that health inequalities can be considered unfair when they are potentially avoidable by organised social action and when the situation that causes poor health is out of the control of those affected.⁶ This includes situations where choices are severely restricted by social, historical, economical or built environments. The causes of health inequity are multidimensional and include factors described as social determinants of health and health sector factors. For the purposes of this paper, we choose to define health equity in this way to reflect our collective interests in social justice and the removal of social barriers posed by discrimination and stigmatisation which affect participation by individuals or populations in opportunities for better health.

Healthcare decisions in most countries are commonly guided by different types of evidence, among which clinical evidence based on randomised controlled trials (‘randomised trials’) usually holds particular prominence. To be useful for health equity policy, considerations for differences within and between groups and gradients of effects across socially stratifying factors are critical. Considering health equity is important to direct approaches that are either universal and intended to address health inequity for a whole population or that are focused and directed at a specific subgroup that experiences health inequity. Improved reporting of health equity in randomised trials is important as some interventions can aggravate rather than reduce inequities (‘intervention-generated inequities’).⁷ A review of public health media campaigns for health behaviour revealed that health information provision was more effective in motivating behaviour change among socioeconomically advantaged groups and widened the observed gap in health behaviour between socioeconomically disadvantaged groups.⁷ Although improving average health is an important social goal, increased inequity is an undesirable side effect. The improved reporting of health equity implications of interventions is critical, and the increased use of randomised trial designs are promoted in the areas of public health⁸ and international development⁹ to build evidence that informs equity-oriented decisions.

Randomised trials provide evidence on the impact of an intervention and, in principle, can also assess their impact on health equity; however, randomised trials often fail to report relevant sociodemographic factors.¹⁰ In addition, subgroup analyses, even when appropriately powered are rarely conducted to assess whether the impact of the intervention differs across subgroups in society (eg, differential impact by socioeconomic status, place of residence).¹¹ To address the poor reporting and analysis of equity impacts in randomised trials, we are assessing the need for additional reporting guidelines for randomised trials that provide evidence about potential impact on health equity.¹² An extension of the internationally recognised Consolidated Standards for Reporting Trials (CONSORT) statement, an evidence-based guideline to encourage completeness and transparency of reporting in randomised trials¹³ is under way. The purpose of this paper is to describe the development and validation of a conceptual framework to identify when a randomised trial provides evidence about health equity. The conceptual framework may be helpful for the design and reporting of randomised trials by trialists, funders, journal editors and other knowledge user groups.

Methods

We convened a collaborative research team and advisory board including interdisciplinary and international knowledge users in a deliberative, consensus building process as part of a larger study described elsewhere.¹² We searched for and did not find any frameworks to define an equity-relevant randomised trial.¹² We then conducted a series of meetings that included representatives of potential knowledge user groups, including: bioethicists, clinicians, funders, journal editors, patient representatives, members of the public, policymakers, systematic review authors and trialists. We engaged in a process to define features of randomised trials that provide evidence about health equity, which we define as ‘health equity-relevant’ randomised trials.

As a first step, we agreed on an operational definition of health equity for our work on health equity in randomised trials, drawing on social determinants of health theory that describes how socially structured health disadvantage is experienced and perpetuated.¹⁴ We agreed, for this context, that the term ‘health equity’ implies that ideally everyone can attain their full health potential and that no one should be disadvantaged from doing so because of their social position or other socially determined circumstance.¹⁵ Health equity addresses both access to healthcare and the broader concept of opportunities to achieve good health.⁶ Moreover, we recognise that there are a range of characteristics of individuals or populations across which health inequities may exist and that reasonable people hold different views about what counts as unfair.¹⁶

Evidence shows that across the social determinants of health for individuals and population groups, people may experience disparities in health status, access to services and the quality of care received.¹⁴ Social determinants may be multifactorial and interact with one another. They can also be dependent on the context and setting, such as the local political or social climate and the form of health systems in which individuals and populations are situated.¹⁷ One organising framework is ‘PROGRESS-Plus’ that depicts social determinants of health and enables the examination of the relationships within and between the multiple, interacting and socially structured characteristics that may influence the opportunities for health of individuals and populations: Place of residence, Raceⁱ/ethnicity/culture/language, Occupation, Gender/Sex, Religion, Education, Socioeconomic status, and Social capital.^18–20 Due to advances in thinking about causal associations/linkages between degree of control in the living environment and socioeconomic inequities in health-related outcomes, ‘Control’ was included with ‘Social Capital’.²¹ PROGRESS-‘Plus’ represents additional context-specific personal or setting characteristics that may be associated with health inequities and that are relevant to our study: (1) individual characteristics, for example, disability, age, sexual orientation; (2) features of relationships with effects on power differentials that impact personal autonomy, for example, children who suffer passive smoking because their parents smoke in the home and (3) time-dependent transitions, for example, migration and/or refugee status.^{22 23} The degree to which PROGRESS-Plus characteristics are associated with health inequity depends on time, place and interaction between the different dimensions.

A conceptual framework was developed and feedback on the concepts and format of the conceptual framework was invited from over 40 clinicians and methodologists in face-to-face meetings, teleconferences and email. These communications used randomised trial examples to structure conversation and debate about the meaning of ‘health equity relevant’. Furthermore, we gathered opinions from many others in local seminars and at international conferences. This feedback was incorporated through iterative adjustments to the conceptual framework.

The conceptual framework was tested with the research team and advisory board.¹² A search for randomised trials was designed with a librarian scientist (JM) in MEDLINE, using text words and subject headings for the concepts of PROGRESS-Plus and a validated filter for randomised trials²⁴ (see online supplementary appendix A). We developed this search by testing whether it identified a reference set of 10 articles classified as health equity relevant according to our framework.¹² The search was run from 2013 to 2015 and generated 4316 hits. Of these, 1025 were judged to be equity relevant based on title and abstract screening. These 1025 articles were arranged in random order, then screened in full text by two independent reviewers for eligibility using the criteria developed to define a health equity-relevant trial. We screened for eligibility until we identified 200 health equity-relevant trials, selected as the sample size for a concurrent methods study (in progress). The sample of trials was tested with the conceptual framework for consistency of concepts. Conversation and debate about concepts occurred and led to consensus around five aspects of randomised trials, and the definition was revised to reflect the consensus reached in these discussions. The research team and advisory board confirmed the final adjustments to the conceptual framework (figure 1).

Figure 1.

Description of the conceptual framework development process.

Results

A randomised trial is considered to provide health equity-relevant evidence if it actually or potentially assesses the effects of an intervention on the health or its determinants of either individuals or a population who experience ill health due to social disadvantage (figure 2). These randomised trials can either exclusively focus on individuals or populations experiencing socially structured disadvantage or consist of a mixed group and collect data potentially useful for assessing differential effects of the intervention across PROGRESS-Plus characteristics (table 1). The evidence from these randomised trials may be used to mitigate socially structured health disadvantage, that is, contribute to understanding of the extent to which the intervention in question is effective in population groups who experience disadvantage. To identify 200 health equity-relevant randomised trials, 643 articles were screened. Screening confirmed that the criteria for health equity relevance could be applied to randomised trials consistently.

Figure 2.

The conceptual framework.

Table 1.

Examples of health equity-relevant studies

Includes individuals or a population that experiences ill health due to social disadvantage (defined across PROGRESS-Plus). Assesses the effects of the intervention on the health or its determinants of either individuals or a population who experience ill health due to disadvantage defined across one or more social determinants of health. Either exclusively focus on individuals or a population or includes a mixed group
‘PROGRESS’ criteria examples		‘Plus’ criteria examples
Assesses effectiveness of the intervention solely for groups who are considered to be at a social disadvantage	Assesses effectiveness of the intervention for groups considered to be at a social disadvantage compared with other groups considered to be at less of a disadvantage	Assesses effectiveness of the intervention solely for groups who are considered to be at a social disadvantage	Assesses effectiveness of the intervention for groups considered to be at a social disadvantage compared with other groups considered to be at less of a disadvantage
A trial of an intervention to prevent domestic violence in Indigenous communities.39	A trial of an informed choice invitation on uptake of screening for diabetes in primary care assessed differences in effectiveness by socioeconomic status.40	A trial that tests a vocational intervention aimed at improving employment for people diagnosed with severe mental illness.41	A trial of a schooling conditional cash transfer programme on the behaviour of young women assessed effectiveness separately based on enrolment status at the start of the study: those who have dropped out versus stayed in school.42

Five aspects of randomised trials are reported as findings and reflect the debated points during the validation of the conceptual framework concepts. We considered five aspects of randomised trials and their relevance to define a randomised trial as health equity relevant: (1) type of interventions, (2) stated objective about equity, (3) estimation of differential impacts, (4) levels of health equity-relevant data and (5) setting and context.

Type of intervention

We agreed that the type of intervention (eg, pharmacological, non-pharmacological, health systems) was not relevant to identifying a randomised trial as health equity relevant. Although health equity is sometimes considered relevant only to public health or upstream interventions, we found many examples of clinical or pharmacological interventions that met our health equity-relevant definition. For example, we considered a randomised trial that evaluated a clinical intervention to promote asthma education for individual inner-city child patients with asthma that improved health outcomes²⁵ to be health equity relevant.

Stated objective about equity

We found that a randomised trial did not need to have an explicit objective about improving health equity to be classified as health equity relevant. For example, a study that implemented a weight loss intervention for native Hawaiians and other Pacific Islanders did a secondary analysis of sociodemographic variables²⁶ but did not identify health equity, inequalities or disparities as part of its objectives. However, the analysis of effects across income groups provides evidence about effectiveness of treatment for a population that is potentially disadvantaged in relation to opportunities for health.

Estimate of differential impacts

For the sake of transparency and to assess whether observed interactions are likely to result from multiple comparisons, it is critically important that an a priori plan of analysis is described that all examined interactions are reported on. Any post hoc subgroup analyses of randomised trial data (eg, across equity characteristics) are at risk of identifying spurious interactions and need to be interpreted with caution. There is, however, increasing recognition that data disaggregation is particularly important, such as analyses across sex/gender.²⁷ The exploration of interactions should be situated within a theoretical perspective²⁸ and/or empirical evidence as a base from which effect measure interactions may be assessed. For health equity, it is important to also consider gradients of effect across social stratifying factors such as income quintiles or with indices that reflect distribution such as the concentration index.²⁹

Many randomised trials include a broad range of population groups, with limited details on sociodemographic characteristics and do not provide disaggregated data that could be used to assess whether effects of interventions differ between groups within the population. Furthermore, some randomised trials use detailed sociodemographic data to adjust analyses for baseline imbalances rather than deliberately exploring whether sociodemographic characteristics modify the effect of the interventions under study, thus losing the ability to assess similarities or differences in effects across these characteristics.³⁰ This represents a missed opportunity for providing evidence about equity.

When trialists do not have a prespecified primary research objective of estimating differential impacts but nevertheless collect data on social determinants of health characteristics, then an opportunity exists to assess subgroup effects. The scope for a randomised trial to contribute health equity data may not be immediately apparent to the randomised trial designer. For example, even if each trial’s statistical power to identify differential effects may be low, combining data from several trials (eg, in meta-analyses) may substantially enhance statistical precision to identify and estimate the strength of such interactions. Data from randomised trials can also be combined with data from observational studies or analysed with methods that incorporate explicit Bayesian priors and/or theories about differential impacts based on well-established bodies of evidence and theory. The problem of multiple tests and the effects on power can be addressed if all analyses of interaction are shown, as well as whether the interaction is defined as departure from ‘multiplicativity’ or ‘additivity’ or both. These interaction analyses should be described³¹ in the randomised trial registration and in the protocol paper. It is our view that opportunities to provide health equity-relevant evidence should be considered up front in randomised trial conception and design and reported transparently. Thus, collecting data about outcomes across social determinants of health may provide useful information for future and additional analyses, now greatly facilitated by global initiatives to allow data sharing.³²

It is equally important to report analyses where no differences in effects are found since these may identify hypotheses for testing in future studies and any failure to identify effects due to low statistical power. If a health programme is equally effective in relative terms, if disadvantaged populations have a higher baseline risk, there will be a greater reduction in absolute events for disadvantaged populations.³³ Such information is valuable for understanding the impact and potential for interventions to improve or exacerbate health inequities.

Levels of health equity-relevant data

According to our conceptual framework, an equity-relevant randomised trial may both focus on individuals and population experiencing socially structured disadvantage and collect data on differential impacts of the intervention in the form of subgroup effects, thus providing more than one level of health equity-relevant data. For example, a randomised trial of a video module that aimed to increase the use of a US poison centre by low-income parents is focused on a population experiencing social disadvantage, but it also provides evidence across language proficiency strata by conducting a subgroup analysis.³⁴ The use of different intervention approaches is complementary and aims to address the multifactorial and intersecting characteristics of populations and individuals that affect health outcomes.

Setting and context

The population characteristics that influence socially structured disadvantage depend on the broader features of setting and context such as social, historical and political systems and require careful consideration. For example, a study with Japanese citizens conducted in Japan would not be classified as an equity-relevant randomised trial unless the participants were also considered to be disadvantaged across another dimension such as sex/gender or income.

We recognise that health equity concerns arise from an array of characteristics; judgments about particular individual or population health determinants are concerns for inequity and may be more difficult because of the need to carefully consider the context of the randomised trial and characteristics, such as age or disability. Studies that report interventions related to overcoming health disadvantage need to consider the relationship between the intervention and how it might relate to equity/inequity, such as an intervention to promote school enrolment and health outcomes between participants who have dropped out versus stayed in school.³⁵

Discussion

Operationalising a definition of a health equity-relevant randomised trial required consideration of broader issues and meanings of social justice informed by values, context and the impact of different social arrangements. Our conceptual framework engages users to consider and identify the many instances of health equity-relevant evidence and the potential contributions to broader societal dialogues. The evidence from these randomised trials may be used to mitigate socially structured health disadvantage, that is, contribute to understanding of the extent to which the intervention in question is effective in population groups who experience disadvantage.

The conceptual framework is proposed as a way to identify and operationalise understandings about how randomised trials can provide evidence about health equity and in particular complements two previous intervention frameworks related to improving health equity. Margaret Whitehead’s typology is based on the underlying theory of change and guides consideration of appropriate application of interventions to address health inequity; interventions fall into one of four main categories: strengthening individuals, strengthening communities, improving living and working conditions and promoting healthy macropolicies.¹⁵ Hilary Graham’s typology identifies three main approaches: addressing health inequity gaps, gradients or targeted approaches.³⁶ Our conceptual framework builds on these classification systems and suggests ways randomised trials can provide health equity-relevant evidence for an intervention, even when providing evidence about equity is not the central aim. Our conceptual framework has potential applications to inform other study designs and is not limited to that of randomised trials.

It is our view that opportunities to provide health equity-relevant evidence should be considered up front in randomised trial conception and design and reported transparently. While different users will hold different definitions and priorities in regard to disadvantage and opportunities for health, we argue that consideration of individual or population characteristics and societal context is of paramount importance. Moreover, consideration of equity aligns with and supports an approach to decision-making that enables consideration of distributional goals and outcomes aimed at addressing health inequity.³⁶ Knowledge about differential effects (or lack thereof) using social determinants of health theory, operationalised across PROGRESS-Plus characteristics can contribute to build knowledge aimed at improved application of effective interventions.

The approach to health equity supported by the conceptual framework promotes thinking about randomised trials as more than just a means of developing evidence about average efficacy or effectiveness. It highlights opportunities for all randomised trials to contribute to a broader social agenda about reducing health inequity. In doing so, it upholds international policy on ‘research for health’ that advocates for the development of research methods to promote better knowledge and standardisation of reporting mechanisms.^{37 38}

Limitations and strengths

A limitation of this approach is that analysing and publishing disaggregated data can be expensive, and it is not desirable for all randomised trials to do subgroup analyses by PROGRESS-Plus characteristics. We propose that, when possible, data on equity-relevant characteristics be collected, even when the study may not have sufficient power to perform subgroup analyses. In this way, data about characteristics linked to health equity may be made available for other researchers to use in pooled analyses, modelling or systematic reviews. Additionally, other knowledge sources/systems are essential to understand health inequities and not all social determinants of health are a source of ‘disadvantage’. As well, relationships within and between social determinants of health mean that there may be a need to explicitly identify the factors that are relevant and therefore have been focused on, as not every facet of inequity is relevant to every research question. The proposed conceptual framework, therefore, is designed to engage users in deliberation about designing and reporting studies that can provide evidence useful to understand distribution of effects, rather than only average effects. Strengths of the conceptual framework are the use of a collaborative approach to development and the iterative testing of the framework with examples of randomised trials.

Conclusion

The conceptual framework defines features of health equity-relevant trials that can be used to design and report both randomised trials and other study designs to improve the evidence base about how to improve health equity. Opportunities to provide health equity-relevant evidence should be considered upfront in trial concept and design. The conceptual framework can provide a stimulus to build knowledge about effects of interventions on health equity and is a first step to improve application of effective interventions and to build evidence that is of greatest relevance and use to individuals and populations.