Abstract
Multilevel interventions are those that affect at least two levels of influence—for example, the patient and the health care provider. They can be experimental designs or natural experiments caused by changes in policy, such as the implementation of the Affordable Care Act or local policies. Measuring the effects of multilevel interventions is challenging, because they allow for interaction among levels, and the impact of each intervention must be assessed and translated into practice. We discuss how two projects from the National Institutes of Health’s Centers for Population Health and Health Disparities used multilevel interventions to reduce health disparities. The interventions, which focused on the uptake of the human papillomavirus vaccine and community-level dietary change, had mixed results. The design and implementation of multilevel interventions are facilitated by input from the community, and more advanced methods and measures are needed to evaluate the impact of the various levels and components of such interventions.
Chronic diseases continue to affect many people in both developed and developing countries. Disparities in the incidence, mortality rate, stage of diagnosis, and risk factors for certain chronic diseases are still evident in many subgroups of the US population characterized by race/ethnicity, income, insurance status, education, occupation, place of residence, and other social factors.1 Many of these disparities have multiple causes, which are influenced by the social determinants of health—the conditions (both positive and negative) in the environments in which people are born, live, learn, work, play, and worship.1
Researchers agree that it is necessary to change negative social determinants to reduce health disparities.2 However, most disparities interventions focus on the individual, often ignoring the person’s social and physical environments.3,4 Not all factors can be modified (such as race and ethnicity), but many can. For example, high rates of breast cancer mortality in South Chicago, Illinois, are caused not only by lower rates of screening mammography in African American residents compared to non-Latino white residents, but also by residents’ poor access to high-quality medical care.5
The elimination or reduction of health disparities requires interventions that address an individual’s social and physical environments, the health care systems he or she accesses, and individual-level factors such as smoking cessation or the use of screening mammography. These interventions must be multilevel—that is, they must occur at a number of levels simultaneously or in close succession. Multilevel interventions target the causes of health disparities by focusing on the following levels of influence that affect health: interpersonal, organizational, community, educational, occupational, environmental, and policy.3
Multilevel interventions may address changes in behavior, such as eating healthier food or taking needed medications; policy changes, such as increasing taxes on unhealthy food or drinks; changes in the delivery of health services, such as addressing a patient’s failure to return for follow-up visits; and environmental changes, such as eliminating food deserts (poor access to affordable, fresh, and unprocessed food in an area) or food swamps (a disproportionate amount of unhealthy food venues and choices in an area) by making healthy food more available at local grocery and corner stores.
Several projects from the Centers for Population Health and Health Disparities—a network of transdisciplinary research centers sponsored by the National Cancer Institute; the National Heart, Lung, and Blood Institute; and the Office of Behavioral and Social Sciences Research—use a multilevel intervention approach to address disparities in cardiovascular disease or cancer. We highlight two interventions, one focused on cancer prevention and the other on cardiovascular disease prevention. The interventions described were based on the model of Richard Warnecke and coauthors, a socioecological model for understanding the multi level contributors to disparate health outcomes.5 This model recognizes the importance of environmental, policy, social, and psychological influences on health.6 We first review challenges in designing multilevel interventions and then discuss challenges and lessons learned from these two interventions.
Challenges In Designing Multilevel Interventions
A variety of study designs can be used to assess the impact of multilevel interventions. One such design, the natural experiment—such as the enactment of the Affordable Care Act7—uses pre-post observations to assess an intervention’s effects.8 Another, the randomized controlled trial, is the gold-standard test of an intervention, but the use of this model to evaluate multilevel interventions is challenging. Selecting who or what will be randomized is a key decision and has implications for sample size, recruitment strategies, mode of data collection, analysis, and how the results and effective interventions are disseminated. Individuals, groups, communities, or neighborhoods can be randomized to receive or not receive an intervention.
The impact of multilevel interventions on an individual may be smaller than that of individually targeted interventions, particularly in five years—the typical time frame for a study. However, the potential reach of multilevel interventions at the population level is much greater than that of individually targeted interventions. Studies have shown that effects at the population level are observed later and increase over time. For instance, in a study in North Karelia, Finland, tobacco use declined over ten years after a communitywide multilevel intervention to reduce cardiovascular disease was launched.9 Intervention designs must consider the time frame during which the intervention will be active and the intervention’s intensity, and measurements must be scheduled at times when important changes in outcomes—such as weight loss—can actually be measured. Large sample sizes with many years of follow-up and data collection are often required to see population-level effects.
Sufficient sample size is also required to allow studies of multilevel interventions to detect and assess possible synergistic effects between and within levels.10 Existing statistical methods and techniques, such as multivariable logistic regression models, make it possible to assess multilevel interventions in the presence of such potentially complex interactions. However, because the existence and nature of such interactions are often unclear, determining the necessary sample size for studies of multilevel interventions can be a challenge.
Examples Of Multilevel Interventions To Address Disparities
The Centers for Population Health and Health Disparities investigated the pathways that result in disparate health outcomes in cancer and cardiovascular disease. The ten centers that were funded collectively conducted nineteen interventions, thirteen of which were multilevel. We highlight two of these projects below.
Cervical Cancer Disparities
The Center for Population Health and Health Disparities at Ohio State University focuses on the reasons for high cervical cancer mortality among women living in Appalachian Ohio. From 2010 to 2015 the center conducted a study to test a multilevel intervention to address the problem of low uptake of the human papillomavirus (HPV) vaccine among girls in the region.11 Research conducted before the implementation of the intervention identified barriers and facilitators to vaccine uptake among parents, clinicians, community members, and young women12 in a setting where there were no public mandates to receive the vaccine and where antivaccine proponents were prevalent. Based on this information, an intervention was developed with input from community stakeholders to focus on parents, clinicians, and health clinics in Appalachian Ohio. Estimates of vaccine uptake and intervention effectiveness were obtained from previous work13–15 to determine sample-size requirements.
A group-randomized trial was implemented in twelve counties in Appalachian Ohio, with six counties randomly assigned to the intervention condition (uptake of the HPV vaccine) and six counties randomly assigned to the comparison condition (uptake of the flu vaccine). The first level of the intervention targeted the health system and included posters and brochures about the HPV vaccine, vaccination reminder cards to be mailed to patients, and buttons about HPV for providers to wear on their lab coats. The goal was to create an environment that supported and encouraged HPV vaccination.
The second level included a PowerPoint presentation on HPV and how to talk to patients about the vaccine, relevant articles on cervical cancer and HPV, and regional HPV statistics—all of which were delivered to providers at regular staff meetings.
The third level focused on parents with daughters needing HPV vaccination. Randomly selected households in each of the twelve selected counties were contacted, and an adult in the household was asked if he or she had a female child of an eligible age who had not been vaccinated for HPV. Eligible and interested parents were enrolled in the study, completed surveys, and were mailed a specially designed DVD about HPV and the HPVvaccine (in the six intervention counties) or a DVD about flu and the flu vaccine (in the six comparison counties), a complementary brochure that reinforced the information in the DVD, and a medical record release form to sign and return to the study office. Parents in the six intervention counties were also called by health educators to address any questions and concerns the parents might have about their daughter’s receipt of the HPV vaccine.
Parents were followed for twelve months. The primary outcome was the receipt—verified by the patient’s medical record—of the HPV vaccine for their daughter within three months of receiving the intervention. Secondary outcomes included receipt of all three HPV shots within twelve months. Changes in medical staff members’ knowledge about and use of health system intervention materials before versus after the intervention were also assessed as secondary outcomes. Additionally, barriers to vaccine receipt reported on the survey by the parent and environmental effects (such as distance to a clinic and county-level factors from census data) were assessed for their effect on receipt of the HPV vaccine.
A total of 337 parents were enrolled. Results indicated that the intervention was successful in increasing the percentage of daughters who received the first shot of the HPV vaccine within three months of receiving the intervention (7.7 percent in the intervention group versus 3.2 percent in the comparison group; p = 0.06). However, the absolute number of girls who received the vaccine, as verified by a review of medical records, remained very low (ten in the intervention group versus four in the comparison group).16 Meanwhile, providers’ knowledge about HPV and the HPV vaccine increased significantly.13
Cardiovascular Disease Disparities
The Center for Population Health and Health Disparities at the University of California, Los Angeles, addressed cardiovascular health among Mexicans and Mexican Americans in East Los Angeles and Boyle Heights. These two neighboring Latino communities in Los Angeles have high rates of obesity-related chronic diseases, particularly diabetes and cardiovascular disease. To understand how environmental, social, cultural, and family factors (such as the home, school, and food environment) affect cardiovascular disease risk, the center conducted three projects with a goal of testing interventions to reduce that risk. The projects used culturally appropriate techniques for community-engaged research.
One of the three projects, the Proyecto MercadoFRESCO (fresh market project), was a multilevel, community-engaged food environment intervention that was intended to improve access to healthy food in the two neighborhoods, both of which were food swamps.17 The project involved multiple stakeholders and intervention targets, including neighborhood residents, business owners, local politicians, high school students, community clinics, community-based organizations, and law enforcement officials. Working with an arts and civics engagement organization and an expert in neighborhood corner stores, the project converted four neighborhood corner stores into stores that carried healthy food, with the goal of improving access to healthy food for the community. Project staff members, corner store owners, and high school students converted the stores by increasing access to fresh fruit and vegetables and conducted a social marketing campaign to increase community-level demand for these foods.
The center developed long-term working relationships with store owners, wholesalers, and farmers markets; reorganized stores so that healthy foods were featured at the front and unhealthy foods at the back; provided new refrigeration and shelving for the new healthy foods; and transformed the overall physical appearance of the stores, including painting and doing general repairs. The multilevel intervention also included communitywide social marketing that emphasized healthy eating and the benefits of shopping locally and learning to eat healthy foods, as well as cooking demonstrations and other community events held at the stores.
Before converting the stores, staff members of the center spent six months evaluating neighborhood stores for suitability, tailoring the store conversion to meet neighborhood demands and context, and working with store owners and local youth to prepare and implement the conversions. For each store, the intervention lasted approximately two years and included ongoing maintenance of the intervention components and owner training, community events to increase patronage and improve healthy eating, and ongoing social marketing and youth engagement.17,18
Store conversions were staggered, and data were collected from two different samples (community residents regardless of whether or not they patronized a converted store and store patrons) immediately before and following store conversions. Baseline data were collected in the period 2011–13, while follow-up data were collected in the period 2012–15. On average, baseline and follow-up data were collected two years apart to allow for sufficient time to determine the effects of the multilevel intervention.
The primary outcomes were changes in the purchasing and consumption of fresh fruit and vegetables at both the individual and community levels. At the community level, perceptions of food availability and the corner stores improved over time, but changes in patronage of the stores and consumption of healthy food were not significant.19 On average, store owners reported a 20 percent increase in sales and high levels of satisfaction with the store transformations.
Discussion
The examples described provide interesting contrasts and speak to the need for policy-level changes to ensure that multilevel interventions have strong impacts. The first study of a multilevel intervention focused on three levels—patient, provider, and clinic—but was conducted in a setting where there was no public policy mandate for receiving the HPV vaccine and where there was a communitywide lack of appreciation for vaccines in general. The effect of the intervention, as measured by uptake of the HPV vaccine among daughters of enrolled parents, was weak.
The second study of a multilevel intervention addressed a topic—healthy eating—that was supported by local politicians from the start and that gained community support. Healthy eating is an especially relevant topic in California, where calories are listed on food menus.20 While the effect of this intervention was also weak, the public generally is more positively oriented to healthy eating than to the HPV vaccine, because of negative perceptions about that vaccine.21,22
While both studies included interventions that influenced multiple levels, neither study was able to achieve the anticipated magnitude of change. Reasons for this could include relatively small sample sizes or a short intervention time period. The lack of success of the interventions suggests that it is insufficient to focus on the individual, interpersonal, organizational, and community levels. Achieving long-lasting, comprehensive improvements to the health and well-being of individuals and communities will likely also require buy-in from politicians and others at the policy level.
In multilevel interventions, the fundamental questions of which factors to address and how to address them are difficult to answer. In the store conversion project, for instance, changes could have been implemented at many other levels (such as marketing or policy) as well, which might have resulted in a greater effect. However, the center staff members felt that the levels chosen were the easiest to address and the most amenable to change.
Conducting multilevel interventions designed to address health disparities is often complex and messy, and the interventions are less likely than individual randomized controlled trials to show strong effects on individuals. Despite the challenges, we maintain not only that multilevel interventions are worth doing, but also that they may be most effective at reducing health disparities. Because such interventions address multiple determinants of health at the same time within complex systems, they may have a broader public health impact among under-served populations, compared to interventions that focus on only one or two levels. Furthermore, when paired with community-based participatory research, which involves communities in identifying and solving their own problems, multilevel interventions seem to be uniquely well suited to reducing health disparities,23,24 as the causes of those disparities are also multilevel.
There is growing awareness of the contributions of multilevel interventions and the importance of engaging diverse community partners to maximize their impact. Such engagement—including increasing support from funders for multilevel interventions—is particularly important when addressing health disparities. For example, the Patient-Centered Outcomes Research Institute has funded a number of studies of multilevel interventions, including one to test such interventions to improve blood pressure control in people who are members of minority groups, have low socioeconomic status, or live in rural areas.25
However, research methodology for studies of multilevel interventions is underdeveloped. In a literature review of what was known about such interventions, Kurt Stange and colleagues found that only a few of the interventions had published results, and the majority were randomized controlled trials focused on prevention, screening, or end-of-life care.10 Most multilevel interventions are contextual (that is, they occur and address issues in just one setting), but the complete context is often not reported, nor are the details of how the interventions were implemented. Most interventions address fewer than three levels, and descriptions of the interventions and measures at the various levels are limited or lacking.10
Stange suggests that multilevel interventions need to focus on design issues (such as moving to pragmatic study designs), analysis (for example, using multimethod approaches and reporting costs and unintended consequences), and the potential for the intervention to be translated to other communities.10 The use of transdisciplinary teams and community involvement are also crucial to the success and implementation of any intervention.10
The Centers for Population Health and Health Disparities have demonstrated that multilevel interventions can be successfully implemented and have an effect on levels beyond that of the individual, though early results have been mixed. Adding policy-level changes to the interventions might result in greater health impacts. Although neither of the interventions discussed in this article specifically addressed policy change, both provide clear evidence of the potential benefit of policies that encourage consumer education in schools and elsewhere about the benefits of HPV vaccines and healthier food options.
Conclusion
The complex determinants of health disparities are not easily studied or addressed. Given that a broad range of biological, interpersonal, organizational, community, environmental, and policy-related factors combine to influence health among disadvantaged populations, research on multilevel interventions is needed. Although intervention research directed at specific individual levels has been conducted for some time, more recent research efforts have attempted to address multiple levels simultaneously to address a single health-related problem or behavior. It is challenging to design a study and measurement strategies that assess the interaction between multiple levels of influence, but doing so is critical. Researchers have not only begun to address these challenges and demonstrate the importance of this research, but they have also identified substantial need for additional methodological development to move the field of health disparities research forward, with the goal of reducing those disparities.
Supplementary Material
Acknowledgments
The authors thank the Centers for Population Health and Health Disparities Multilevel Interventions to Address Health Disparities Writing Group and funders. For a complete list of writing group members and funders, click on the Appendix link in the box to the right of the article online. The views expressed in this article do not reflect any official position of the National Cancer Institute or the National Heart, Lung, and Blood Institute.
Contributor Information
Electra Paskett, Professor of internal medicine at the Ohio State University College of Medicine, in Columbus.
Beti Thompson, Professor in the Cancer Prevention Program at the Fred Hutchinson Cancer Research Center, in Seattle, Washington.
Alice S. Ammerman, Professor of nutrition in the Gillings School of Global Public Health and director of the Center for Health Promotion and Disease Prevention, both at the University of North Carolina at Chapel Hill
Alexander N. Ortega, Professor in and chair of the Department of Health Management and Policy, Dornsife School of Public Health, at Drexel University, in Philadelphia, Pennsylvania
Jill Marsteller, Associate professor at the Johns Hopkins Bloomberg School of Public Health, in Baltimore, Maryland.
DeJuran Richardson, Professor of mathematics at Lake Forest College, in Lake Forest, Illinois, and an adjunct professor of biostatistics at Rush University Medical Center, in Chicago, Illinois.
NOTES
- 1.Murray CJ, Kulkarni SC, Michaud C, Tomijima N, Bulzacchelli MT, Iandiorio TJ, et al. Eight Americas: investigating mortality disparities across races, counties, and race-counties in the United States. PLoS Med. 2006;3(9):e260. doi: 10.1371/journal.pmed.0030260. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Marmot M, Wilkinson RG, editors. Social determinants of health. 2. Oxford: Oxford University Press; 2005. [Google Scholar]
- 3.Sallis JF, Owen N, Fisher EB. Ecological models of health behavior. In: Glanz K, Rimer BK, Viswanath K, editors. Health behavior and health education: theory, research, and practice. 4. San Francisco (CA): Jossey-Bass; 2008. pp. 465–86. [Google Scholar]
- 4.Institute of Medicine. Health and behavior: the interplay of biological, behavioral, and societal influences. Washington (DC): National Academies Press; 2001. [PubMed] [Google Scholar]
- 5.Warnecke RB, Oh A, Breen N, Gehlert S, Paskett E, Tucker KL, et al. Approaching health disparities from a population perspective: the National Institutes of Health Centers for Population Health and Health Disparities. Am J Public Health. 2008;98(9):1608–15. doi: 10.2105/AJPH.2006.102525. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Stokols D, Grzywacz JG, McMahan S, Phillips K. Increasing the health promotive capacity of human environments. Am J Health Promot. 2003;18(1):4–13. doi: 10.4278/0890-1171-18.1.4. [DOI] [PubMed] [Google Scholar]
- 7.Tetuan TM, Ohm R, Herynk MH, Ebberts M, Wendling T, Mosier MC. The Affordable Health Care Act annual wellness visits: the effectiveness of a nurse-run clinic in promoting adherence to mammogram and colonoscopy recommendations. J Nurs Adm. 2014;44(5):270–5. doi: 10.1097/NNA.0000000000000066. [DOI] [PubMed] [Google Scholar]
- 8.Pande AH, Ross-Degnan D, Zaslavsky AM, Salomon JA. Effects of healthcare reforms on coverage, access, and disparities: quasi-experimental analysis of evidence from Massachusetts. Am J Prev Med. 2011;41(1):1–8. doi: 10.1016/j.amepre.2011.03.010. [DOI] [PubMed] [Google Scholar]
- 9.Puska P, Nissinen A, Salonen JT, Toumilehto J. Ten years of the North Karelia Project: results with community-based prevention of coronary heart disease. Scand J Soc Med. 1983;11(3):65–8. [PubMed] [Google Scholar]
- 10.Stange KC, Breslau ES, Dietrich AJ, Glasgow RE. State-of-the-art and future directions in multilevel interventions across the cancer control continuum. J Natl Cancer Inst Monogr. 2012;2012(44):20–31. doi: 10.1093/jncimonographs/lgs006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Reiter PL, Katz ML, Paskett ED. HPV vaccination among adolescent females from Appalachia: implications for cervical cancer disparities. Cancer Epidemiol Biomarkers Prev. 2012;21(12):2220–30. doi: 10.1158/1055-9965.EPI-12-0850. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Katz ML, Reiter PL, Heaner S, Ruffin MT, Post DM, Paskett ED. Acceptance of the HPV vaccine among women, parents, community leaders, and healthcare providers in Ohio Appalachia. Vaccine. 2009;27(30):3945–52. doi: 10.1016/j.vaccine.2009.04.040. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Perkins RB, Zisblatt L, Legler A, Trucks E, Hanchate A, Gorin SS. Effectiveness of a provider-focused intervention to improve HPV vaccination rates in boys and girls. Vaccine. 2015;33(9):1223–9. doi: 10.1016/j.vaccine.2014.11.021. [DOI] [PubMed] [Google Scholar]
- 14.Aragones A, Bruno DM, Ehrenberg M, Tonda-Salcedo J, Gany FM. Parental education and text messaging reminders as effective community based tools to increase HPV vaccination rates among Mexican American children. Prev Med Rep. 2015;2:554–8. doi: 10.1016/j.pmedr.2015.06.015. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Parra-Medina D, Morales-Campos DY, Mojica C, Ramirez AG. Promotora outreach, education and navigation support for HPV vaccination to Hispanic women with unvaccinated daughters. J Cancer Educ. 2015;30(2):353–9. doi: 10.1007/s13187-014-0680-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Paskett ED, Krok-Schoen JL, Pennell ML, Tatum CM, Reiter PL, Bernardo BM, et al. Results of a multilevel intervention trial to increase human papillomavirus (HPV) vaccine uptake among adolescent girls. Cancer Epidemiol Biomarkers Prev. 2016;25(4):593–602. doi: 10.1158/1055-9965.EPI-15-1243. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Ortega AN, Albert SL, Sharif MZ, Langellier BA, Garcia RE, Glik DC, et al. Proyecto MercadoFRESCO: a multi-level, community-engaged corner store intervention in East Los Angeles and Boyle Heights. J Community Health. 2015;40(2):347–56. doi: 10.1007/s10900-014-9941-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Sharif MZ, Garza JR, Langellier BA, Kuo AA, Glik DC, Prelip ML, et al. Mobilizing young people in community efforts to improve the food environment: corner store conversions in East Los Angeles. Public Health Rep. 2015;130(4):406–15. doi: 10.1177/003335491513000421. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Ortega AN, Albert SL, Chan-Golston AM, Langellier BA, Glik DC, Belin TR, et al. Substantial improvements not seen in health behaviors following corner store conversions in two Latino food swamps. BMC Public Health. 2016;16(1):389. doi: 10.1186/s12889-016-3074-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Center for Science in the Public Interest. California first state in nation to pass menu labeling law [Internet] Washington (DC): The Center; 2008. Sep 30, [cited 2016 Jun 28]. Available from: http://cspinet.org/new/200809301.html. [Google Scholar]
- 21.Fowler EF, Gollust SE. The content and effect of politicized health controversies. Ann Am Acad Pol Soc Sci. 2015;658(1):155–71. [Google Scholar]
- 22.Kelly BJ, Leader AE, Mittermaier DJ, Hornik RC, Cappella JN. The HPV vaccine and the media: how has the topic been covered and what are the effects on knowledge about the virus and cervical cancer? Patient Educ Couns. 2009;77(2):308–13. doi: 10.1016/j.pec.2009.03.018. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Gorin SS, Badr H, Krebs P, Prabhu Das I. Multilevel interventions and racial/ethnic health disparities. J Natl Cancer Inst Monogr. 2012;2012(44):100–11. doi: 10.1093/jncimonographs/lgs015. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Holmes JH, Lehman A, Hade E, Ferketich AK, Gehlert S, Rauscher GH, et al. Challenges for multilevel health disparities research in a transdisciplinary environment. Am J Prev Med. 2008;35(2 Suppl):S182–92. doi: 10.1016/j.amepre.2008.05.019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Patient-Centered Outcomes Research Institute. Comparative effectiveness of health system vs. multi-level interventions to reduce hypertension disparities [Internet] Washington (DC): PCORI; [cited 2016 Jun 28]. Available from: http://www.pcori.org/research-results/2015/comparative-effectiveness-health-system-vs-multi-level-interventions-reduce. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.