Designing and Assessing Multilevel Interventions to Improve Minority Health and Reduce Health Disparities

Abstract

Multilevel interventions can be uniquely effective at addressing minority health and health disparities, but they pose substantial methodological, data analytic, and assessment challenges that must be considered when designing and applying interventions and assessment.

To facilitate the adoption of multilevel interventions to reduce health disparities, we outline areas of need in filling existing operational challenges to the design and assessment of multilevel interventions. We discuss areas of development that address overarching constructs inherent in multilevel interventions, with a particular focus on their application to minority health and health disparities. Our approach will prove useful to researchers, as it allows them to integrate information related to health disparities research into the framework of broader constructs with which they are familiar.

We urge researchers to prioritize building transdisciplinary teams and the skills needed to overcome the challenges in designing and assessing multilevel interventions, as even small contributions can accelerate progress toward improving minority health and reducing health disparities. To make substantial progress, however, a concerted and strategic effort, including work to advance analytic techniques and measures, is needed.

Investments in biomedical, social, and behavioral sciences have greatly expanded knowledge of the etiology, prevention, detection, and treatment of many diseases, and many of these advances have contributed to improved population health. Nonetheless, not everyone has benefited equally from scientific development and medical advancement,¹ and strategies to reduce socioeconomic and racial/ethnic disparities frequently remain elusive. In large part, this is because the factors that underlie disparities are wide ranging and multifaceted; these factors encompass every socioecological level, including individual, interpersonal, community, and societal (in this supplement issue, Alvidrez et al. describe the National Institute on Minority Health and Health Disparities [NIMHD] framework).²

NIMHD defines a health disparity as a health difference that adversely affects disadvantaged populations on the basis of 1 or more of the following health outcomes:

• 1.higher incidence or prevalence of disease,
• 2.earlier onset or faster progression,
• 3.poorer daily functioning or quality of life,
• 4.premature or excessive mortality, and
• 5.greater global burden (see Duran and Pérez-Stable in this supplement issue).³

Health disparities are generally embedded in social or structural determinants of health, which cannot be effectively addressed at the individual level alone (see Brown et al. in this supplement issue).⁴ Efforts to reduce disparities and promote health equity must therefore address the dynamic interplay of multiple levels of influence to be effective.⁵ Indeed, social characteristics such as race/ethnicity and socioeconomic status are highly interconnected with critical influences across levels, such as the physical environment, access to health care services, and psychosocial experiences rooted in socioenvironmental factors (e.g., discrimination, victimization, stress).

A critical limitation of many interventions targeting minority health and health disparities is that they have tended to focus on single or individual-level factors. As a result, the National Institutes of Health has called for more research employing a multilevel approach to address the health needs of disadvantaged populations.⁶ To respond to this call, we must consider approaches for systematically designing and assessing the efficacy of multilevel interventions in these populations. This will enhance researchers’ ability to develop efficient and effective approaches, identify multilevel causal factors contributing to intervention efficacy, and understand their dynamic interactions within and across intervention levels. Progress is being made in this regard, with a growing body of work demonstrating the promise of multilevel interventions for improving health outcomes in disadvantaged populations and addressing health disparities.^7,8

However, designing and assessing multilevel interventions continue to pose substantial methodological and data analytic challenges⁹ that may hinder their use. Some challenges may be universal and occur in all types of multilevel interventions, whereas others are specific to interventions addressing minority health and health disparities. Continued growth in the use of multilevel interventions to reduce health disparities depends on the ability of researchers to effectively address the challenges of designing and implementing rigorous outcome assessments. To facilitate the adoption of multilevel interventions to reduce health disparities, we outline areas of need to fill operational challenges in the design and assessment of multilevel interventions: both universal challenges inherent in all multilevel interventions and challenges specific to interventions for minority health and health disparities (see the box on this page).

Recommendations on Research Priorities for Multilevel Interventions to Reduce Health Disparities.

Methodological Considerations	Research Priorities
Framework and design	Develop and evaluate novel and systemic theoretical frameworks and models that can account for interactions among multiple levels and that include socioeconomic and environmental determinants of health disparities.
	Integrate conceptual frameworks addressing issues specific to minority health and health disparities into multilevel intervention research.
	Develop new metrics that incorporate elements of existing theoretical frameworks that can be applied to multiple levels to understand independent and interactive effects of intervention components and confounding variables as well as elements such as fit and longevity of interventions in health disparities–relevant contexts.
	Evaluate the effectiveness of various multilevel research designs (e.g., randomized controlled trials, quasiexperiments, natural experiments) to assess potential for determining mechanisms that are associated with health disparities in populations of interest.
Constructs, measures, and intervention components	Develop reliable and culturally and linguistically appropriate constructs and measures to examine intervention components at each level to provide a comprehensive understanding of intervention outcomes on health disparities, and evaluate new and existing measures among minority and underserved populations.
	Identify specific, individual intervention components (e.g., through pretrial designs, multiple iterative pilot tests) that are active in shaping outcomes and have been shown to produce reductions in health disparities.
	Assess interactions between intervention components across levels to identify situations in which they either impede each other or produce synergistic effects to allow effective replication and scaling.
Statistical and analytic approaches	Employ the requisite advanced analytical tools (e.g., hierarchical linear modeling, structural equation modeling) and aggregation approaches to properly detect health outcome changes across levels, potential interactions between factors, delayed effects, and intervention effectiveness to reduce health disparities.
	Analyze process variables to better understand the causal pathways and mechanisms that affect intervention outcomes, contextual factors associated with variation in outcomes, and ways to optimize intervention effects to reduce health disparities.
	Analyze the temporal effects of policies in the socioecological context and how they influence outcomes across all levels, as well as their impact on reducing health disparities.

We argue that overcoming both types of challenges is essential for promoting the use of multilevel interventions to address minority health and health disparities. Rigorous approaches to trial design and assessment will improve the utility of multilevel interventions by identifying mechanisms or pathways to effectively reduce health disparities. To advance the field, we discuss areas for development that address overarching constructs inherent in multilevel interventions, with a focus on application to minority health and health disparities. This approach provides the most useful information to researchers because it integrates information related to health disparities research into the framework of broader, more familiar constructs.

ADVANTAGES OF MULTILEVEL INTERVENTIONS

In conceptualizing multilevel interventions, “levels” are defined by various socioecological models and usually include larger environment, or “upstream,” factors and more individual biological, or “downstream,” factors that lie closer to individual health outcomes.¹⁰ One conceptualization of this framework is that of Warnecke et al.,¹¹ who identify social conditions and policy influences as the upstream factors and biological and genetic pathways as the downstream factors, with social and physical context and individual demographic factors in the middle (Figure 1).

FIGURE 1—

Conceptualization of a Framework Identifying Social Conditions and Policy Influences as Upstream Factors and Biological and Genetic Pathways as Downstream Factors, With Social and Physical Context and Individual Demographic Factors in the Middle

Source. Adapted from Warnecke et al.¹¹

Multilevel interventions specifically require action targeting 2 or more levels of influence at the same time or in close temporal proximity. The intervention approaches implemented at each level typically vary in type (e.g., behavioral, environmental, health systems, or policy) and may be delivered simultaneously or progressively over time or settings. Importantly, intervention effects on 1 level can, and likely will, interact with those at other levels, for example, through synergistic effects (i.e., effects greater than the sum of the individual component effects).¹² Therefore, interventions aimed at minority populations that assess both the direct and interactive effects across levels can help identify the multiple causes (e.g., social and physical environments and individual determinants) that affect minority health and health disparities.

Our adaptation of the Warnecke et al.¹¹ model in Figure 2 illustrates how a multilevel intervention can be used to address smoking among African American men in the United States. African American men are an important population for public health intervention because they experience poorer lung cancer survival and later stage at diagnosis than do other racial or ethnic groups.¹³ The model shows that certain individuals have greater genetic susceptibility to the effects of smoking, addiction to nicotine, or biological preference for menthol, and some of these genetic factors vary by race and ethnicity.¹⁴ Smokers with these biological traits often smoke more and inhale more deeply, thus drastically increasing their risk of developing lung cancer.¹⁵ The model shows how biological susceptibility to smoking can be facilitated by upstream factors, such as family members, friends, and co-workers who smoke¹⁶; an overabundance of tailored tobacco marketing in African American neighborhoods¹⁷; workplaces that provide easy access to smoking areas and smoke breaks¹⁸; health care providers who fail to offer cessation services¹⁹; and residence in states with low tobacco taxes. All of these factors reinforce norms in favor of smoking behaviors²⁰ and are more prevalent in African American communities.^17,19,20

FIGURE 2—

Illustration of How a Multilevel Intervention Can Be Used to Address Smoking Among African American Men in the United States

In a multilevel intervention, family members, friends, or co-workers could be included in a cessation program, and multiple options for program delivery could be used. In the health care system, health care organizations and providers could uniformly ask about smoking status, advise smokers to stop smoking, and refer all smokers to cessation services. At the environmental and policy levels, governments could ban menthol or limit marketing of tobacco; stores could be incentivized to replace tobacco ads; local ordinances could be enacted to limit the size and number of ads, thus reducing environmental cues to smoke²¹; nonsmoking laws could be better enforced or enacted in workplaces with on-site cessation programs offered²²; and taxes could be increased to deter the purchase of tobacco products.²³ This example covers a breadth of potential causes and possible intervention activities focusing on minority populations that are complex and likely beyond the scope of any single multilevel intervention. However, it illustrates the multitude of potential targets that lie within and between levels that could be the target of a multilevel intervention designed to reduce health disparities.

CHALLENGES AND APPROACHES

Multilevel interventions offer the promise of substantial impact for reducing health disparities; however, studies on these multilevel interventions inherently have considerable methodological challenges. We focus on study design and assessment. Although ethical issues are beyond the scope of this article, it is worth noting that multilevel interventions involve unchartered territory for ethical considerations, especially relating to minority health and health disparities. A paradigm shift is needed to assess potential risks and harms to distal intervention targets (e.g., effects of an intervention on a community as a whole, not just the individual participants).

To date, multilevel interventions, including those addressing minority health and health disparities, have been limited in several ways. Researchers rarely clearly articulate a theoretical framework for their multilevel interventions²⁴ and often select less rigorous study designs,²⁵ both of which impede assessment efforts. Most multilevel interventions address only 2 (e.g., patient and family) or sometimes 3 (e.g., patient, family, and community) levels, and primary outcomes are often assessed only at the individual level.²⁴ Further, because many study designs deliver all intervention components as a package, it is difficult to know which components affect which outcomes and in which combinations.

Other limitations include weak analytic plans, inadequate sample sizes, and the use of statistical approaches that do not account for the complexity of data across levels or potential interactions between levels. These challenges often arise because of the lack of transdisciplinary teams needed to design and evaluate multilevel interventions and the lack of sufficient time and resources. To ensure that multilevel interventions to reduce health disparities result in the best possible research outcome, essential methodological elements must be considered (see the box on page S87).

Framework and Design

Framework.

Theoretical frameworks should guide study design, the selection of appropriate measures, and intervention development through the identification of leverage points that address the root causes of health disparities. Several conceptual frameworks designed for guiding health disparity and minority health research illustrate the ways concepts such as racism, discrimination, cultural factors, and racial segregation contribute to chronic disease. For example, the public health critical race praxis, which is grounded in critical race theory²⁶ explores the concepts of race, racism, and structural inequalities to understand the causes of health disparities and determine strategies for eliminating them. Incorporating the concept of racism allows a comprehensive examination of the way racism can affect health care delivery, health behaviors, and chronic disease. Similarly, the lifespan biopsychosocial model of cumulative vulnerability and minority health posits that race/ethnicity and socioeconomic status interact to predict health outcomes by shaping exposures to psychological adversities through cognitive–emotional, biobehavioral, and biological pathways.²⁷

The concentric circle model (an adaptation of the ecodevelopmental model) posits that understanding the social–cultural level (e.g., ethnic–cultural values, racially discriminatory social polices), the community level (e.g., institutional racism), and the familial level (e.g., differential acculturation) can provide insight into the relationships that cause health disparities when designing culturally relevant interventions.²⁸ Although conceptual frameworks offer critical perspectives relevant to health disparity research, few studies have used them to guide multilevel interventions.

Theory-based frameworks and models also can provide clearly defined metrics to assess study outcomes at each level and suggest potential mechanistic factors to measure in evaluation. For example, the comparative mixed-methods participatory evaluation model posits that outcome metrics should include acceptability, social–cultural validity, integrity, outcomes and impact (both intended and unintended), sustainability, and institutionalization.⁹ Importantly, this model explicitly directs researchers’ attention to evaluating intervention aspects, such as social and cultural validity and sustainability, that are critical for interventions conducted in minority and underresourced communities. Without the application of such frameworks, researchers may be less likely to assess the fit or identify pertinent leverage points and potential longevity of the intervention in the specific socioeconomic climate of implementation.

Study design.

Investigators also must consider the suitability of the study design for yielding maximally informative results. Although the use of randomized controlled trials is common, in community settings, randomized controlled trials may not be feasible because investigators have less control over intervention delivery (a potential threat to fidelity), use of usual care control groups may be viewed as unethical, and resistance to randomization may be heightened in ethnic minority communities. Several alternatives to randomized controlled trials that are more context specific and maintain scientific rigor have been suggested for evaluating community programs.²⁹ Tradeoffs have been explored among various study designs that are applicable to translational research and, thus, evaluation of multilevel interventions tested in health disparity populations.³⁰

Other study designs (e.g., quasiexperiments, natural experiments, and time series designs) can be well suited for multilevel health disparities research. For example, natural experiments are ideal when studying how well policy reduces health disparities, and this study design can accommodate methodologically rigorous evaluation approaches and may be a particularly pragmatic approach in underresourced environments. Natural experiments can provide insights into the social determinants of health, such as environment and policy effects, which are major causes of health disparities and for which traditional randomized experiments may be difficult to conduct. They have advantages for external validity because they reflect the real-world challenges of implementing programs and policies that cannot be ensured in the unusual circumstances of community trials or effectiveness studies.³¹ Finally, it is important to engage community representatives as active decision-makers in selecting the appropriate study design and assessment methods to increase the potential to identify leverage points and reduce health disparities.

Constructs, Measures, Intervention Components

Construct selection.

The careful selection of constructs and measures is crucial for the optimal assessment of multilevel interventions to address health disparities. The widespread application of reliable, culturally valid, and linguistically appropriate measures in studies allows comparisons between studies and facilitates meta-analysis. However, many patient- or participant-reported outcome measures are geared toward English speakers who have relatively high literacy and numeracy levels.³² Because of the wide disparities in health as a result of socioeconomic status and language, a critical need is to identify, develop, and deploy measures that are appropriate for even very low literacy and numeracy populations. In the context of multilevel interventions, this consideration takes on additional importance because of the dearth of appropriate, validated outcome measures at the family, community, structural, and other levels. Indeed, literacy and language barriers and other social issues persist at these more upstream levels in the socioecological framework and also tend to cooccur within and between levels in the framework (e.g., low literacy at the individual level in a neighborhood with low socioeconomic status in a region that is medically underserved).¹⁰ Constructs and measures that assess various leverage points at each intervention level can identify salient (individual and socioenvironmental) factors that cause health disparities and suggest ways to bring about change.

Identification of active components.

Refining interventions requires researchers to identify intervention components that are the most efficacious, as well as those that are not making a measurable difference in outcomes. This is an especially important endeavor for health disparities–focused interventions, as they are often implemented in resource-poor environments. Multilevel interventions tend to be expensive because, by definition, they require implementing activities across levels. Therefore, identifying active components that reduce health disparities can allow more effective replication and scaling because researchers can focus on ensuring fidelity for the most critical pieces of the intervention. In addition, intervention adaptation is a key consideration for health disparities–focused interventions. Translation of interventions across health disparity populations will require adaptation of intervention components to fit the contextual and cultural realities of a particular group. Because of the limited availability of resources in health disparity populations and because of competing needs, identifying the most critical components of an intervention will allow researchers to focus adaptation efforts on the most effective elements.

The multiphase optimization strategy (MOST)³³ lays the groundwork for gathering empirical evidence on how each component functions relative to the outcome and how components function together interactively. This strategy involves conducting factorial experiments to determine the individual and interactive influence of intervention components. For example, the MOST design was used to build a remotely delivered parenting intervention for low-income mothers to prevent obesity among their children.³⁴ The pilot study was successful in assessing the feasibility and acceptability of 8 intervention components and identified particular components associated with lower study completion rates. Researchers, then, could refine intervention components before implementing a larger study to prevent childhood obesity in low-income populations. However, for large, fully powered efficacy trials, investment in this process requires obtaining considerable upfront resources, which may be challenging, especially when working with underresourced communities.

Multiple, iterative pilot tests that employ both qualitative and quantitative methods can be used to ensure that the study components proposed are realistic and achievable with the specific minority population and in the specific community identified. Researchers have additionally suggested conducting usability tests of intervention components in which the components are iteratively refined. Usability testing can provide preliminary evidence of intervention efficacy on proximal outcomes elements and determine whether intervention components can be administered with sufficient fidelity.³⁵ This is critical for small sample populations (e.g., American Indians, Pacific Islanders, sexual gender minorities), whose needs and responses may be different from majority populations.

Components across levels.

Evaluating interactions between components at each socioecological level can identify situations in which they either impede or enhance each other, thus allowing evaluation of the components’ effects in light of the entire study design. Certainly, for optimal efficiency and cost effectiveness within underresourced communities, it would be important to know whether a smaller set of components could achieve a similar measure of effect. One example of such an assessment can be seen in the Improving Access to Mental Health in Primary Care study in older and minority ethnic populations³⁶; this was an intervention with 3 components of demonstrated efficacy, each operating at a different socioecological level: community engagement, primary care, and tailored psychosocial interventions. However, the sample size was small, and so evaluations focused on process rather than outcomes variables.

This intervention illustrates the reality that even though evaluation of interactions across socioecological levels is frequently recommended, the resources and sample size required to evaluate these interactions may be substantial. As a result, these quantitative assessments are rarely done. Much may be gained, however, from channeling resources into multilevel interventions with factorial designs so that the performance of active components can be evaluated both independently and together; this would optimize interventions for resource-poor environments that are common in the context of minority health and health disparities.

Particularly important for health disparities interventions, a mixed-methods approach can capture interactions among components and allow deeper participant and organizational perspectives to emerge. Ethnographic and qualitative methods, such as participant observation, key informant interviews, and ethnographic surveys, offer multiple unique perspectives to understand the degree of interaction between and within the levels of the intervention. These methods also provide insights into potential confounding variables, unintended consequences,⁹ and which interactions should be prioritized and operationalized for further assessment. In the context of multilevel interventions to reduce health disparities, this is crucial because of the potential for unique perspectives, experiences, and social pressures of minority populations to underlie unanticipated interactions between intervention components. For example, the pairing of a health care–based component and a community-based component may heighten intervention efficacy among a majority population. However, this combination may undercut the effectiveness of both components in some minority populations if mistrust in the health care system bleeds over into mistrust of other intervention components.

Statistical and Analytic Approaches

Statistical approaches.

Selecting the appropriate statistical and analytic approaches is crucial for assessing multilevel interventions, especially those aimed at addressing minority health and health disparities. The assessment of intervention outcomes across multiple socioecological levels is an inherently complex undertaking and requires special statistical considerations. For example, the statistical challenges associated with analyzing data from multilevel interventions include analyzing outcomes at each targeted level (e.g., individual, community, policy) and examining mediators and moderators involved in the pathways of change. In this type of research, observations often lack independence because of correlation or clustering of data (e.g., individuals are nested in communities; communities are nested in cities). Researchers need to employ analytical approaches that can account for these relationships (e.g., hierarchical linear modeling, structural equation modeling) or aggregation procedures to appropriately assess outcomes.²⁴

For quasiexperimental studies, propensity score analysis can be used to control for potential confounding of covariates and reducing bias in effect estimates.¹² In natural experiments, this method can be used to balance the distribution of social determinants (e.g., neighborhood characteristics) across race/ethnicity to effectively evaluate the effect of a policy on eliminating health disparities. In health care delivery interventions, dynamic simulation modeling (e.g., system dynamic modeling) can be used to capture complex interactions between patients, providers, and system-level components.³⁷

In addition, researchers should employ methods to ensure sufficient power to detect interactions among intervention components at different levels. However, power calculations depend on previous knowledge of how the primary outcome and predictor variables within and between the levels affect one another, and such information may be unknown.¹² For example, research on a multilevel intervention targeting clinic and patient levels to increase mammography screening rates in Latinas found that there were increased rates of mammography screening among patients receiving the promotora-led intervention but that the study was underpowered to examine clinic-level effects or moderators of efficacy.³⁸ The development of rigorous analytic plans that account for data complexity, change over time, and potential interactions across levels is critical for improving multilevel intervention assessment in health disparities research.

Considering and analyzing intervention processes.

Considering and measuring the intervention process is essential because of the likelihood of uncontrolled and unpredictable change in contextual variables within and across levels during a multilevel intervention study. This is especially true in minority health and health disparities contexts, where intervention-relevant factors (e.g., transportation, access to medical care) can be less stable. One useful approach to address such instability is sequential multiple assignment randomized trials with an adaptive, randomized design,³⁹ which allows a varied response to the individual or the environment. Iterative pilot tests can tell researchers whether adaptive tailored variables and their associated evaluation will be needed. Evaluation requires close monitoring and careful documentation of process variables to better understand the outcomes generated. For example, capturing whether the intervention dose was delivered (provided) versus received (taken) is a process measure. For instance, resources (e.g., healthy foods) may be provided as part of an intervention, but in resource-poor communities, these may be more likely to be shared with one’s family as opposed to being consumed by the intended individual. Without detailed knowledge of process, the pathway to the outcome result is unclear.

Relatedly, the empirical assessment of intervention fidelity is less widespread than is ideal.⁴⁰ Therefore, even if the contribution of individual components is known, unless fidelity is assessed, it is unknown whether participants received intervention components as designed. Maintaining fidelity, as well, may be more difficult in low-resourced settings that have many competing health and welfare priorities. Adequate planning and assessment are also crucial with respect to participant reach and retention, given potential added challenges with minority and low-income samples (e.g., distrust, housing stability). Substantial financial and research team expertise resources are required to achieve optimal sample composition and longevity. Designing and implementing approaches to collect detailed process data throughout the multilevel intervention allow post hoc analyses of mediating factors operating within and across multiple levels.

Temporal trends.

Multilevel interventions are complicated by real-world events and secular trends that are outside the control of the research team. Communities and societies are dynamic systems that interact over time, changing and reacting to circumstances (e.g., political climates, funding of federal, state, and local health programs). Moreover, changes in any of these systems often do not have a direct cause and effect relationship but instead can have delayed effects on processes and outcomes. In addition, health disparities tend to take time to manifest and thus to assess because they necessarily involve relative outcomes between advantaged and disadvantaged groups; so many interventions tend to target minority health outcomes of a single group. Thus, capturing temporal trends and delayed outcomes can be challenging, especially when targeting constructs at the upstream levels (e.g., policy, community); however, it is these upstream levels that can effect the most change. Indeed, crucial determinants of disparities often lie within upstream ecological levels. For example, research on the implementation of the Affordable Care Act captured temporal trends in access to health care, revealing significant reductions in uninsured rates among African American and Hispanic populations and a narrowing of health disparities.⁴¹

To effectively address health disparities through multilevel interventions, greater attention to temporal trends is needed. Because of the socioeconomic, environmental, and institutional barriers that contribute to health disparities, long-term infrastructure and institutional support can allow researchers to target these relative factors and outcomes at multiple levels to effect meaningful and lasting change.

FUTURE DIRECTIONS

Health disparities are pervasive throughout society—in part because of complex interactions between behavioral, social, economic, and environmental factors—and have significant societal consequences that affect the nation’s health. Multilevel interventions that target the broader community, societal, and environmental contexts and that affect individual behavior or risk produce greater reductions in health disparities than do approaches targeting a single level.²⁵ Research to assess the effectiveness of multilevel interventions that can measure the complex interplay between the levels is urgently needed.

We have identified and discussed several challenges associated with designing and assessing multilevel interventions that require attention from the health disparities research community. These challenges necessitate transdisciplinary partnerships (e.g., biomedical, behavioral, and social scientists) for the specific purpose of achieving a common scientific goal. These transdisciplinary teams could specifically advance analytic techniques and measures for multilevel interventions. In so doing, the mechanisms and pathways that effectively reduce health disparities among different levels and maximize intervention success and sustainability can be more fully understood. Future research on multilevel interventions should focus on developing and refining guidelines for systematically designing and assessing the efficacy of multilevel interventions to improve minority health and reduce health disparities.