The ecology of multilevel intervention research

Abstract

Behavior change research to promote health and prevent disease increasingly relies on a complex set of interacting characteristics across levels of influence such as biological, psychological, behavioral, interpersonal, and environmental. How to best develop health-related interventions that incorporate the individual, the macro-environment, and their interactions remains a challenge. This article considers a set of key dimensions that constitute what we refer to as the ecology of research across a broad context of multilevel research (MLR), spanning fundamental multilevel research (FMLR), multilevel intervention research (MLIR), and multilevel implementation science (MIS). With the goal of promoting improvements in MLIR, we describe the inherent interdependencies among aspects of research and consider how the growth and development of evidence and resources influence the cross-talk among researchers from different perspectives (e.g., disciplines and domains). We propose a framework that highlights opportunities to reduce barriers and address gaps in areas critical to generating an evidence base through MLR, MLIR, and MIS. Overall, we aim to support strategic decisions that can accelerate our understanding of ML health outcomes and interactions among factors within and across levels, with the goal of strengthening the effectiveness of ML interventions across health-related outcomes.

Development of strategies to support healthy lifestyle choices benefits from research that considers influences from the individual to the environment and includes many different perspectives.

Implication

Practice: Practitioners who want to maximize the impact of health behavior change interventions should implement findings from interventional research that spans multiple levels—from the biological, psychological, and behavioral through interpersonal and environmental contexts.

Policy: Effective health care policy requires an understanding of how context—including interpersonal, neighborhood, built environment, and systems-level contexts—can influence an individual’s ability to adopt and maintain behaviors that prevent disease and promote optimal health.

Research: Structured approaches that guide knowledge sharing among researchers across domains can facilitate strategic decision making and accelerate advances in multilevel research.

INTRODUCTION

The adoption and maintenance of behaviors that enhance health and prevent disease involves complex sets of interacting factors across levels of influence, such as biological, psychological, behavioral, interpersonal, and environmental [1–4]. The embeddedness of individuals within sociocultural contexts, as emphasized in consensus government reports such as Healthy People 2020, highlights the importance of creating environments that promote good health for all Americans [5]. Research has shown that health disparities may be created and maintained when complex interactions across multiple levels of influence are not fully considered [6]. Recognizing that health behaviors are influenced by contextual factors—such as the cultural norms and policies within health systems and communities—provides the opportunity for developing, testing, and implementing interventions that target multiple levels of intervention. But how to best develop health-related interventions that incorporate the individual, the macro-environment, and their interaction remains a challenge.

Social and behavioral health researchers can help address this challenge through strategic emphasis on multilevel intervention models and methods. This approach will better equip researchers to address social determinants of health and advance population health beyond individual and short-term outcomes. However, the capacity to understand behavioral phenomena is limited by the available evidence as well as the degree of readiness within a domain to incorporate the most sophisticated approaches. For instance, a relatively robust literature across health domains illuminates mediators of behavior change at the individual level, whereas evidence at higher levels is scant [7]. Therefore, to comprehensively intervene on key health-related behaviors, it is necessary to more consistently and strategically expand the evidence base beyond single levels of analysis through programs of research that systematically measure or intervene at multiple levels (see Fig. 2). In addition, better integration, descriptions, and testing of theories and methods—that guide how to influence change at higher levels of the socioecological model and elucidate how these levels interact to promote positive and sustained behavior change—can accelerate research advances.

Fig 2.

| Exemplar features across the continuum of fundamental multilevel research to multilevel intervention research.

Improvements in multilevel research (MLR) will be advanced by considering the ecology of research across a broad context from fundamental multilevel research (FMLR) to multilevel intervention research (MLIR) and on to multilevel implementation science (MIS). In this article, we describe the inherent interdependencies among areas of research and consider how the growth and development of evidence and resources influence cross-talk among researchers from different perspectives (e.g., disciplines and domains). Our proposed framework highlights opportunities to reduce barriers and address gaps in areas critical to generating an evidence base through MLR, FMLR, MLIR, and MIS. We aim to support strategic decisions that can accelerate our understanding of interacting ML factors and enhance the effectiveness of ML interventions across health-related outcomes.

INTERDEPENDENCE ACROSS MULTILEVEL RESEARCH FROM THEORY TO IMPLEMENTATION

Behavioral scientists recognize that inherent reciprocal, cyclical, and iterative relationships exist across the continuum of MLR, including multiple levels of behavioral theory, fundamental research, interventions, and implementation, each of which provides feedback loops for advancing and refining science (see Figs. 1 and 2). The value of theory is predicated on linking each of these relationships in meaningful ways to contribute to the iterative process and ultimately guide interventions. For instance, moving beyond general frameworks to theories that specify relationships across levels requires methods to understand how to operationalize relevant contextual features and measure interactions between individuals and the environments within which they are embedded [8]. As ML theories become more specific, strategies are needed to test, refine, and translate them using statistical and computational approaches designed to account for the complexities of MLR (c.f., [9]).

Fig 1.

| Reciprocal, cyclical, and iterative relationships between multilevel research, theory, intervention, and implementation.

For intervention research to fully benefit from evolving ML theories, methods are needed to develop interventions that correspond to the relevant theory. Despite more than a decade of research developing strategies to map intervention techniques to theoretical elements in the individual behavior change domain, there continues to be limited guidance on how to consistently achieve these direct links [10, 11]. Therefore, a lack of correspondence between the constructs and intervention components during the intervention development and statistical analyses phases in research studies persists [12, 13]. This lack of correspondence reduces the theoretical precision of intervention research, which further contributes to inconsistent effects of evidence-based interventions. In addition, advances to FMLR and theory refinement are forestalled when research studies focus on reporting intervention results without analyses of the mechanisms of action. Likewise, opportunities exist for MLIR to more consistently incorporate implementation measures of multiple interventions across systems, which in turn can contribute to an accumulating knowledge base of barriers to success and possible synergistic effects [14].

CONCEPTUAL AND DEFINITIONAL VARIABILITY ACROSS MLR DOMAINS

A panoply of definitions and conceptual descriptions associated with MLR and MLIR are found across the scientific literature and National Institutes of Health (NIH) funding opportunities. The differences can stem from variations in a range of dimensions associated with degree of maturity and sophistication of the MLR across domains (see Table 1), which are influenced by philosophical and epistemic origins, developmental trajectories, and pragmatic considerations. These sources of influence can introduce confusion that hinders the success of cross-disciplinary collaboration and slows scientific progress. For instance, differences in definitional and conceptual perspectives may stem from the historical traditions and epistemological origins of a discipline or health domain (e.g., the problem-based orientation of public health vs. the individual-focused orientation of psychology). Further, from a funding agency perspective, pragmatic considerations associated with filling research gaps may result in nuanced domain specific definitions. Additional examples of such distinctions were seen during a series of roundtable workshops focused on MLR, hosted by the National Cancer Institute (NCI) at the 2018 Annual Society of Behavioral Medicine meeting. Discussions among approximately 50 leading experts highlighted ongoing debates regarding the feasibility of developing and the importance of using ML theories and whether ML interventions are needed for all health outcomes and in all contexts. The discussions highlighted the apparent differences in perceptions of the needs and future directions for MLR and MLIR and how such differences are heavily influenced by vantage points within particular programs of research, disciplinary orientation, or health domains.

Table 1.

| Framework of key dimensions and guiding questions to accelerate advances in MLR across domains

Key dimensions of the ecology of MLR that highlight areas of potential variability across domains	Guiding questions that can help identify the profiles of progress for health behavior research domains and illuminate opportunities for cross-talk based on relative strengths and weaknesses across domains.
General characterization of the research domain
Duration of identifiable programs of research	How long has the research domain been active? (e.g., tobacco cessation, 50+ years; HPV vaccination, <10 years)
Historical origins of the domain	Which fields or disciplines have influenced the philosophical or epistemological orientation of the extant research (e.g., a problem-based orientation of public health vs. an individual-focused orientation of psychology)? How might that influence the level of MLR in the domain?
Disciplines/fields involved/engaged	Is the research in the domain typically conducted within only one field of study? Do a few disciplines dominate the extant research?
Levels of analyses and degree of integration across levels
Integration across disciplines	Do disciplines in this field/domain regularly collaborate?
Degree of research at each level of analysis	Is there research in this domain at all levels of analysis? Have some levels of analysis received less attention than others? Which levels, and why?
Degree of FMLR	Do studies in this domain regularly include more than one level of analysis? (e.g., see Fig. 1 for additional considerations) Are certain levels of analysis more frequently included in FMLR in this domain? Which levels, and why?
Integration and interactions across levels	Is the body of research occurring predominantly in disciplinary silos or within levels of analysis? Has there only been limited integration across levels? Do studies regularly assess interactions across levels of analysis? Which levels, and why?
Degree of MLIR	Do studies in this domain regularly include interventions at more than one level? (e.g., see Table 1) Are certain levels of analysis more frequently included in MLIR in this domain?
Degree of specificity, complexity and use of theory, conceptual models, and frameworks
Use of theory, conceptual models, and/ or frameworks at each level of analysis	Are theories, conceptual models, and/or frameworks regularly used to inform research in this domain? In particular, are well-specified theories regularly used to inform research? Are theories used more regularly at one level or another?
Degree of ML theory, conceptual models, and/or frameworks	Do theories, conceptual models, and/or frameworks in this domain connect multiple levels of analysis? Is theory regularly used to inform FMLR/MLIR? Are frameworks and conceptual models used to inform FMLR/MLIR?
Complexity and specificity of ML theory, conceptual models, and/or frameworks	Does the domain primarily characterize the relationships between key domain elements through broadly construed frameworks (e.g., socioecological model)? Are the theories, conceptual models, and/or frameworks domain specific? Do studies include well-specified relationships among variables/constructs and across levels?
Alignment of methods, measures, and intervention development with MLR goals
ML methods	What methods are typically used in the domain? Are the methods appropriately matched to the type of ML data and research questions in the domain?
Measures	Do measures exist to adequately characterize the phenomenon in question? What is the quality of measurement? Does that quality vary across levels of analysis? Which levels, and why?
ML intervention development	Do interventions in the domain typically include more than one level of analysis? Are the interventions informed by ML theory, frameworks, or models?
Availability of organizational, institutional, sociopolitical supports for MLR
Funding	Is the MLR in the field primarily investigator driven or are there formal NIH office/units or dedicated programs of funding (e.g., center initiatives funded via RFAs)?
Education and training	Do education programs addressing this domain offer courses related to ML research, theory, and/or methods? Are there courses/workshops regularly available to support training on ML topics?
Sociopolitical context	Are there broad social trends or ethos that may hinder or foster intervention efforts (e.g., broad understanding in USA regarding dangers of smoking)? Are there long-standing interventions in place that may influence MLIR in the domain (e.g., tobacco taxes, quitlines)

FMLR fundamental multilevel research; HPV human papillomavirus; MIS multilevel implementation science; MLIR multilevel intervention research; MLR multilevel research; RFA research funding announcement.

To overcome debate among researchers and conflicting definitions and/or divergent concepts, health scientists will benefit from establishing core principles across MLR. Consensus around cross-cutting principles can serve as the foundation for dynamic frameworks flexible enough to account for variability in available evidence of, and approaches used to conduct, MLIR across health domains and serve to facilitate the transfer of knowledge across disciplines and domains [15]. Convening scientists across disciplines and domains to develop a shared understanding of divergent views and begin to identify common ground can help accelerate the communities’ shared MLR goals of understanding which intervention level(s) to intervene on, when, and for whom.

Those who wish to identify core principles and common ground, as well as unique challenges, gaps, and opportunities across domains may include scientists working together to advance a domain or field, establish research agendas, or develop new research programs or projects. Facilitation of a broad discourse around MLR or more focused discussions among collaborating scientists can benefit from structured approaches that guide perspective taking [16] and enhance critical awareness [17]. Communities or groups of scientists guided by questions in Table 1 might start by considering overarching questions such as the following: What factors have influenced the level of maturity of my field/domain? For instance, has there been decades of research or is it a newly emerging area of research? Have certain fields dominated the research in this domain and if so, how has that influenced the type of methods employed? What are the strengths and weaknesses of the domain? How can I best advance the field/domain? Who (from what disciplines) can help advance new directions or approaches? What other fields/domains may have knowledge/lessons learned to help advance my field/domain? Using an objective approach to clarify the relative strengths and weaknesses across domains can help illuminate the conceptual and definitional differences, identify potential lessons learned and transferable knowledge across domains, and highlight next steps.

KEY DIMENSIONS OF THE ECOLOGY OF MLR

We propose that the ecology of MLR is comprised of dimensions across five categories: (a) general characterization of the research domain; (b) levels of analyses and degree of integration across levels; (c) degree of specificity, complexity, and use of theory, conceptual models, and frameworks; (d) alignment of methods, measures, and intervention development with MLR goals; and (e) availability of organizational, institutional, and sociopolitical supports for MLR. Table 1 highlights dimensions that influence the base of evidence for MLR and thereby highlights key features and leverage points for research policies and practices that can serve to accelerate progress across the continuum of MLR. For instance, we assert that the level of evidence available across MLR and the accumulation of knowledge may be used to inform ML theory and, furthermore, that these will vary depending on the degree to which ML measurement and interventions occur. More specifically, across the FMLR to MLIR continuum, the consistency in which ML studies, at minimum, measure factors at more than one level, and, as possible, intervene at one or more levels, and/or examine interactions across levels (as described in Fig. 2) will influence the robustness of the corpus of evidence available to inform the development and refinement of ML theory (as depicted in Fig. 1). In addition, this example highlights how features of MLR introduce potential confusion among researchers regarding what researchers may consider to be FMLR and/or MLIR (e.g., as depicted in Fig. 2). Increasingly complicated groupings of these features can illuminate the interdependent and dynamic nature of MLR, with unique combinations serving to generate new knowledge that builds on and supports the evolution and maturity of MLR. This example also highlights strategic opportunities to identify methods and study designs that regularly support the generation of knowledge to inform full-scale multilevel intervention trials.

The subsequent sections of this article provide suggestions for supporting progress of MLIR, with the aim of encouraging institutions, funding agencies, and researchers to consider the implications of the unique combinations of existing evidence and resources and the importance of knowledge sharing and lessons learned across areas of research.

VARYING LEVELS OF EVIDENCE AND RESOURCES ACROSS FEATURES CREATE UNIQUE PROFILES

The framework highlights key dimensions of the ecology of MLR. The guiding questions can help researchers describe features of these key dimensions for a given health domain. Taken together, we propose these features together form a unique profile of the domain, which will change over time and influence the way in which MLIR develops. Additional contextual variables may also interact to influence the needs of a given domain. For example, with more than 50 years of research [18] involving a wide range of disciplines, tobacco use research has evolved over time. Despite decades of research, in the late 1990s, the domain continued to lack integration across disciplinary-based programs of research. In response, the National Cancer Institute made significant investments to facilitate transdisciplinary research [19], which changed the availability of multilevel evidence across several key features.

With its rich history, the tobacco-control field offers unique opportunities to consider the conditions under which multilevel interventions are needed. Recent studies have demonstrated strong support for the effectiveness of policy interventions for the prevention and control of tobacco use [20]. This has prompted debates on whether policy interventions should be given greater attention or support, and whether they supersede the need for individual-level interventions across behaviors [21]. Yet, new tobacco policies are being introduced after decades of individual-level research in addition to extensively resourced and widely disseminated comprehensive tobacco-control programs that include public service campaigns, organizational strategies in the healthcare system, worksites, and schools. These programs also include federal, state, and local infrastructure that deploy individual-level interventions such as quitlines and physician counseling, which have reached millions of individuals in the USA over several decades [22, 23]. In the tobacco use domain over the course of decades, interventions have been deployed across the nation at multiple levels (e.g., tobacco taxes, quitlines) and thus can be considered a giant multilevel natural experiment, which offers several lessons that can be applied to other health behaviors. Furthermore, considerations regarding the need for the development of ML interventions and methods for studying ML factors, following decades of interventions implemented across society at all levels of analysis, will be very different for tobacco use research than for other domains.

A research area, such as human papillomavirus (HPV) vaccine uptake, introduces a different profile of features across key dimensions due to its history of less than 10 years of intervention research, the delivery of the vaccine to youth with consent from parents in the health care context, and public debate regarding the safety and acceptance of the HPV vaccine (e.g., [24]). For instance, the inherent multiple-level nature of obtaining the HPV vaccine often requiring the engagement of youth, parents, physicians, and the health care system for individuals to receive the vaccine may increase the saliency ML theory and interventions in this area. Overall domains, each with their own profile, will have their own set of challenges and opportunities, related to their unique MLIR-related histories and profiles of progress, which can influence the discourse among scientists across domains. The use of a structured dialog approach can help distinguishe unique features and illicit potential common ground to effectively establish the most pressing research needs within and across domains.

PARAMETERS FOR MULTILEVEL INTERVENTION RESEARCH FROM A BEHAVIORAL SCIENCE PERSPECTIVE

Designing MLIR requires a spectrum of configurations that cut across levels and requires particular attention to conditions may vary across research domains and implementation contexts. MLIR typically targets factors at two or more different levels of analysis, with specific constructs that can be measured and evaluated at multiple levels. Interventions should be feasible and sustainable, and at least one intervention should address broader social- and system-level variables that may affect the outcomes. Furthermore, variations may include a hybrid MLIR design. For instance, in tobacco-control research, a clinical trial may be combined with a natural experiment (e.g., a smoking policy is introduced in a population exposed to an existing individual-level intervention) such that only one intervention may be delivered by researchers, yet both intervention effects are considered. Differences exist concerning whether ML intervention require modes of delivery at different levels (e.g., at the individual patient level through an online self-administered behavior change program and at the provider level by a physician through physician counseling) or if the intervention targets need to involve multiple levels (e.g., an intervention that targets an individual and a systems-level factor such as health insurance reimbursement) [25].

Parameters for what qualifies as MLIR are complex. Accumulated knowledge will enable us to determine what intervention is needed, when, and for whom. In some contexts, competing demands from one level can thwart efforts at another level because there are political, social, and technical barriers surrounding implementation of interventions. Furthermore, not all organizations can or want to have change, so multilevel interventions may only be effective when there is engagement from multiple levels. Instead of generating potentially limiting parameters to MLR, developing typologies and/or frameworks of MLIR configurations can better enable research synthesis and facilitate lessons learned within and across types of MLIR.

DESIGNING AND EVALUATING MULTILEVEL INTERVENTION APPROACHES

Developing multilevel interventions involves a phased approach that begins with identification of an important clinical or public health question as well as application of theory and frameworks as well as findings from epidemiologic, qualitative, and basic behavioral and social sciences research (e.g., [26–28]). Ideally, available evidence can provide a foundation for understanding the behavior or health condition not only at the individual level but also at some combination of individual, interpersonal, organizational, community, and societal levels. Once such evidence exists and can be supported by theory, frameworks, or models, development of interventions can build on this foundational knowledge through selection, design, refinement, and ultimately testing of strategies to address change at multiple levels. This requires a careful examination of roles needed to create change at each level; understanding of the determinants of the behaviors or structures requiring change; selection of the most appropriate behavioral, social, organizational, community and/or policy targets for intervention; and identification and testing of strategies to influence them.

Questions to be addressed in multilevel intervention research include the following: Are the levels interacting, or do they compete? How do they interact? How can they be disentangled? Using qualitative, experimental, and observational methodologies can be helpful in teasing out the interactions between the different levels. Multilevel interventions are aided by recent expansion of available intervention designs, such as SMART designs adapted for the multilevel context. Frameworks such as the Medical Research Council (MRC) framework [29] can be useful for explicating the steps in planning and conducting multilevel intervention research. Tools such as Intervention Mapping [29] and the Behavior Change Wheel [10, 11, 30] can facilitate the identification of high-impact leverage points, drivers of behavior at multiple levels, and organization, community, policy, environmental, and structural elements that present both barriers to be addressed and targets of opportunity for intervention. Whether a single-level (e.g., ORBIT; [26–28]) or multiple-level (e.g., MRC framework; [29]) framework is used, critical questions to ask during the early phases of multilevel intervention development include the following: Who are the agents of change? What are the levers or drivers of change that can be used as intervention targets at each level? At every level there are different motivations and set of barriers and facilitators that must be understood to be influenced. Use cases and examples of successful multilevel intervention research studies can help inform decision making regarding levels, intervention components, and targets. Using tools such as systems science approaches can help identify appropriate targets for intervention.

COMPLEXITY AND SPECIFICITY OF ML FRAMEWORKS, MODELS, AND THEORIES

Confusion regarding distinctions among theories, conceptual models, and organizing frameworks leave researchers conflating the terms (e.g., defining theory as a conceptual or organizing framework) or using multiple terms to describe the same work [8]. We consider the degree of specification of relationships among factors to distinguish a theory from a model or framework. For instance, theory would specify explicit directional relationships among all noted constructs and indicate testable hypotheses within and across levels. Alternatively, a framework or model would delineate general associations between lists of variables or groups of broadly defined constructs. Multilevel theories can help illuminate clear, specific hypotheses and research questions across and between conceptually distinct levels to guide MLR and interventions, whereas frameworks can elucidate complex systems and explanatory factors, often without specifying targets and allowing for a broad outcome measure. The current state of the science for multilevel health behavior research primarily includes general conceptual models and frameworks, with fewer elucidated multilevel theories.

The feasibility of developing multilevel health behavior theories and the optimal degree of specification for multilevel frameworks, models, and theories is deliberated in the field, with the utility of broad generalizability weighed against the precision of theories developed to address specific domains, populations, or contexts. Ongoing debates are fueled by the difficulty of developing theories, particularly when the quantity and quality of our existing research evidence across levels of analysis and health domains are uneven or limited. The development of robust theories depends on the cumulative knowledge generated by both “top-down” and “bottom-up” approaches. “Top-down” evidence derived from broader approaches such as the socioecological model helps to inform specific models, whereas “bottom-up” streams of study-specific models may be synthesized to inform more generalized models or theories. The iterative process of developing and refining frameworks and models to inform theory highlights the need to support dynamic theory development [31]. The complex nature of generating theories within a particular level of analysis is compounded by the challenge of also accounting for relationships between levels. If ML theory generation is the goal, moving beyond broad indications of cross-level relationships to delineating highly specified interrelationships across levels will require strategic efforts to fill in evidence gaps, systematic efforts to synthesize and apply cumulative knowledge, and collaborative efforts to integrate knowledge across levels of analysis and health domains [15].

The accumulation of evidence associated with interrelationships among factors is needed to support theory generation, yet often models do not adequately specify relationships between factors, which limits the testing of associated hypotheses. In a recent review of NCI grant proposals for cancer-screening interventions, all proposals included study-specific frameworks or conceptual models (though not necessarily ML) [13], yet few provided well-specified relationships among the theoretical constructs and even fewer included analysis plans to examine “mechanisms of effect or conditions necessary for intervention effectiveness” [13]. Without such analyses, new knowledge to improve subsequent behavioral interventions and the theories used to guide them is not generated. Models lacking specification may limit the potential scientific and theoretical value of a given study to contribute to the cumulative knowledge-based needed to advance MLIR; guidelines for creating well-specified study-specific models and appropriate methods for analyzing relationships would help foster ML theory development and improve future MLIR.

Multilevel interventions potentiate synergistic change, whereas interventions at one level may produce unintended change at another; the breadth of possible influences across levels create difficulties for optimal theory development. Emerging research areas may lack the foundational evidence and methods for analyses to specify and test dynamic interplay across levels that is needed to inform theory development. Nonetheless, progress demands refining or developing new theoretical or conceptual models to understand the hypothesized interacting processes within and across multiple levels and over time. We encourage researchers to provide experimental evidence of the relative importance of intervening on one level over others and on how interventions at different levels may have additive or competing effects.

METHODS FOR ASSESSING INTERACTIONS ACROSS LEVELS IN FMLR AND MLIR

Qualitative, experimental, and observational methodologies can all be helpful in identifying interactions between multiple levels and address questions such as the following: Are interventions effects across levels interacting, or do they compete? How do mechanisms of action interact? How can they be disentangled? However, we lack knowledge about which methodology is the best empirical test of multilevel theories and frameworks and which methods are best to translate hypotheses into appropriate analytic models. Emerging approaches such as multilevel factor analysis, multilevel structural equation models, and dynamic computational models [8, 9], as well as approaches that mix these methods, can help to generate hypotheses for how levels interact. These approaches can also be useful for the development of multilevel theories (theory generating and testing; e.g., [32]).

Context is integral to the design and evaluation of the external validity of multilevel interventions [33] to support generalizability to other behaviors, populations, and settings. Given the importance for explaining behavior and shaping behavior change, standards for evaluating contextual variables as potential moderators that facilitate or impede an intervention’s effectiveness would advance the field. Natural experiments are a great opportunity to understand in real time how changes within the social, neighborhood, and policy environments interact and influence behaviors and to provide insights into the effects that programs, interventions, or policies have on health-related outcomes across multiple levels [34]. The development of networks of schools, worksites, institutions, and communities that can be quickly accessed can help leverage opportunities as they arise if existing guidelines were available to shape rapid assessments.

Assessment of an intervention includes the main effects of the intervention, as well as the interactions across levels. Weiner et al. discuss the importance of developing an approach and analysis plan that includes identification of how the levels and interventions will interact and propose a useful typology to guide the analysis of interactions across levels [35]. To take advantage of the complexity of working across levels and within systems as well as the opportunities to capture large volumes of data that exist today, we need to provide greater support for more sophisticated approaches/methods such as computational models, system dynamic approaches, and tests of moderation/mediation. Interdisciplinary research programs may more easily integrate novel approaches by harnessing existing resources within a complementary field of study. For instance, programs of research that engage computer scientists or system engineers to develop interventions within a health care system may have researchers and data systems that can more readily incorporate computational modeling approaches. Considering these factors that facilitate novel analytic approaches will be important when developing funding or training strategies to advance MLIR.

USE OF TECHNOLOGY TO SUPPORT FMLR AND MLIR

The rapid acceleration of technological advances offers opportunities to gather new types and volumes of data previously unimaginable. The use of real-time and fine-grained data collected through ecological momentary assessment methods, sensors, and other tools will help advance our understanding of behavior, their time course, and the reciprocal relation between behaviors and other variables such as those at other levels [36]. To further advance the field, technology-mediated integration of multiple data streams and bidirectional flow of information (e.g., real-time data capture and system-level data) requires accessible systems with adequate attention to privacy and security, as well as enhanced usability.

Given the high costs of measurement, identifying opportunities to leverage existing systems, databases, and platforms (e.g., surveillance systems and natural experiments) can help improve efficiency/feasibility of multilevel research. As the development and use of such infrastructures increase, the establishment of standard measures across platforms can facilitate data integration and knowledge synthesis thereby accelerating the generation of a cumulative evidence base [37, 38]. Convening researchers across disciplines/levels and stakeholders/“owners” of these types of platforms via consensus conferences or crowdsourced discussions (e.g., NCI’s Grid-Enabled Measure platform) to determine quality of measurement can further identify candidate measures for standard/broad use.

Multiple methods may include qualitative, quantitative, geospatial, and systems engineering approaches and techniques within one study. Focusing the design and delivery of the intervention with dissemination and implementation in mind will enhance the real-world context and ensure key decision maker and stakeholder perspectives are included [39, 40].

QUALITY OF ML MEASURES

For multilevel interventions, examining cross-level effects (e.g., the effect of a family on an individual, and individual on a family) introduce unique challenges at each level of analysis. Furthermore, MLIR requires development of constructs and measures that can capture the interaction across levels and enable researchers to understand how time, space, and life course interact with the intervention effects. Measurement issues may be especially complex for MLIR, requiring particular attention to independence, reliability, validity, sample size, and power [41, 42].

Advances in measurement are needed to improve dyadic-, interpersonal-, social-, and community-level measures that can capture context, relationships, and interactions/dynamics beyond the typical use of aggregated individual-level measures to represent higher level constructs. Methods such as multilevel confirmatory factory analysis help develop measures that are appropriate for using at the group level and help the researcher understand how the measure is functioning at both individual and group levels.

Some major measurement challenges for MLIR include identifying what level should be targeted for interventions and what factors should be measured at the various levels to understand the effect of the intervention on behavior. One way to address this is to take a community-engaged approach, where key stakeholders are involved to cocreate interventions and help identify the most salient and sustainable metrics [43]. Doing so may allow for the interventions to then be more consistent with the real world in which behaviors occur and facilitating faster uptake, adherence, and, ultimately, broader implementation across contexts.

CROSS-DISCIPLINARY TEAM SCIENCE APPROACHES

Integration of stakeholder perspectives and engagement beyond the research community

Developing multilevel interventions that meet the needs of populations and will be taken up within particular contexts require the knowledge and engagement of those they are intending to serve. Funding agencies can facilitate engagement of representatives from health care systems and communities by explicitly supporting and encouraging the integral involvement of patients (e.g., PCORI), physicians, and community stakeholders. Approaches such as community-based participatory research and methods to facilitate the coproduction of knowledge [44] provide guidance for involving stakeholders throughout the research process including problem formulation and intervention development. Citizen scientists can serve as integral members of a research team and aid in data collection, hypothesis generation, interpretation, and provide unique contextual knowledge [45]; by doing so, they can help alleviate data collection and cost burden of MLIR and most importantly provide insights into local areas of need. Strategies to incentivize cross-disciplinary approaches should be developed to help reduce barriers to establish partnerships, increase breadth of engagement, and thereby enhance the relevance and impact of the intervention.

Integration across disciplines

Klein et al. [15] offer recommendations for designing studies to increase the capacity to subsequently integrate knowledge and share lessons learned across behavioral science domains, which can serve to accelerate MLR. To conduct MLR, cross-disciplinary teams are needed; researchers may have differing philosophical and epistemological assumptions that stem their disciplinary orientation or other features associated with the ecology of their research domain [46]. For instance, a psychologist may value theory more than an epidemiologist, whereas an anthropologist may prioritize context higher than a computer scientist. Collaborating across such boundaries takes skill and resources often not learned within traditional academic programs. In addition to individual perspective taking or structured dialog using the guiding questions presented in this paper, work from the science of team science can be helpful in informing team development, training, and the conduct of research where diverse disciplines are represented as is typically the case in multilevel research [47, 48]. Tools such as a collaboration plan or collaborative agreements can help prevent conflicts [49]. The use of workshops (e.g., ideas labs) designed to foster cross-disciplinary, cross-level collaborations, or funding mechanisms (e.g., center initiatives) that include adequate support for coordination and enable researchers to leverage resources (e.g., measures, data) across projects can enhance the success of MLR [47, 50, 51].

Innovative approaches to support MLIR

Multilevel intervention research is challenging due to the complexity inherent in these types of studies. This complexity is based on the time needed to develop and test interventions that may be different at different levels, the cost to conduct and evaluate such interventions, the time and resources needed to conduct such research, including developing and sustaining partnerships with stakeholders such as institutions, state and local government entities, clinical networks, and communities. The significant cost of MLIR associated with developing interventions and preparing to conduct research (e.g., designing interventions, preliminarily testing them or optimizing them for each level, securing funding, developing partnerships) and conducting the studies (cost of implementing interventions and collecting data across levels) may preclude or at least discourage researchers from conducting such research. Limits in funding can make it difficult to design and implement studies with sufficient power to test the effects of multilevel intervention components at group, organization, and community levels. These challenges require innovative solutions, including novel ways to access existing resources and infrastructures to facilitate multilevel intervention research. One idea is to create networks of schools, worksites, institutions, and communities that can be quickly accessed to realize opportunities as they arise. By funding transdisciplinary groups of investigator–community–organizational partnerships, existing resources can be leveraged and used as needed with new, facile funding mechanisms to facilitate the design and conduct of multilevel trials that move away from funding Principal Investigators (PIs) to funding networks of patients, providers, clinics/hospitals, neighborhoods, and communities.

TRAINING RESEARCHERS TO ADVANCE MULTILEVEL INTERVENTION RESEARCH

To advance MLIR, the scientific community needs to be adequately trained to develop and use ML theory and frameworks, employ appropriate methods and measurement, collaborate across disciplines, and engage stakeholders throughout the research process. Training efforts should build on existing training models on relevant multilevel methods (e.g., NIH-sponsored training in randomized controlled trials, implementation science, health behavior theory). A key strategy for MLIR includes the development of a new intensive summer advanced training institute (ATI). An ML ATI can provide interactive training that integrates advanced training in the areas of multilevel theory, frameworks, methods, and measures as applied to the development and testing of ML interventions. Such courses can reach across career levels and include opportunities for grant development training with multidisciplinary teams, mentor–mentee assignments, train-the-trainer models, etc. Online access to didactic webinars and other ML training curriculum content can serve to supplement in-person activities and expand reach. Online components can be used as prerequisites to in-person training and provide support to university instructors. Furthermore, efforts to embed MLIR training into curriculum from pre- or post-doc (F31/T31/K) through mid-career levels (PIs) and mid-career training (retraining) can help address the limits of the current workforce. Opportunities to bring training to researchers across the full spectrum of career levels can also be achieved via the inclusion of trainings at scientific conferences.

CONCLUSION

Researchers, funding agencies, and policy makers increasingly recognize the value of MLR, both in terms of scientific output and as an effective and efficient use of resources to enhance health and wellness. Yet challenges associated with science policy and insufficient training and infrastructure continue to impede progress. Multilevel research is complex and requires substantial investments in time and resources, which may be at odds with university (e.g., promotion and tenure) and agency (e.g., funding, review) practices and policies. Advances in multilevel research may be hampered by the lack of robust building blocks such as data, methods, theory, funding, and cross-disciplinary collaborations. For instance, challenges of using extant data—such as access to data, the burden of data collection, privacy, and integration—hinder the accumulation of evidence to support the development of hypotheses across levels and facilitate comparative tests between multilevel and single-level interventions.

We introduce the ecology of MLR, including key dimensions and guiding questions, and propose that taking a macro-level view of MLIR across health domains can effectively determine where, when, and how best to support MLIR. This can be accomplished by implementing systematic frameworks and processes for supporting iterative reflection among researchers, funding agencies, and policy makers regarding the implications of the key features of MLIR. These strategies can help shift ongoing dialog from debates fueled by conflicting definitions, concepts, and approaches to generative dialog centered around common principles and transferable lessons across disciplines and health domains to accelerate advances in MLIR.

Compliance with Ethical Standards

Funding: None.

Conflicts of Interest: All authors declare that they have no conflicts of interest.

Ethical Approval: This article does not contain any studies with human participants performed by any of the authors.

Informed Consent: This study does not involve human participants and informed consent was therefore not required.

Welfare of Animals: This article does not contain any studies with animals performed by any of the authors.