Leveraging group model building to operationalize implementation strategies across implementation phases: an exemplar related to child maltreatment intervention selection

Abstract

Background

Implementation strategies can help support the adoption and implementation of health interventions that are appropriate for a local context and acceptable to decision makers and community members. Implementation strategies should be designed to handle the complexity of the multi-level, dynamic contexts in which interventions are implemented. Systems science theories and methods explicitly attend to complexity and can be valuable for specifying implementation strategies. Group Model Building (GMB) combines research partner engagement strategies with systems science to support researchers’ and partners’ learning about complex problems and to identify solutions through consensus. This paper specifies how GMB can operationalize implementation strategies — methods for supporting evidence implementation in real-world practice — and describes how GMB can aid in selecting and tailoring both health interventions and implementation strategies. A case study in child maltreatment prevention planning is provided to illustrate how GMB was used to specify the “actions” — strategy activities — for three implementation strategies (conduct local consensus discussions; build a coalition; model and simulate change) during the earliest implementation phases, with the goal of supporting intervention selection decisions. Examples are provided of generalizable research products that can be produced concurrently through GMB, in addition to contextually-driven implementation support.

Methods

Participants (n = 8) were engaged over four sessions using tailored GMB activities. Participants generated a qualitative system dynamics model that described their theory of change for how to prevent child maltreatment in their communities. This theory of change reflected a dynamic understanding of the interconnected determinants of child maltreatment.

Results

GMB was acceptable to participants and resulted in products that could be used for implementation planning (e.g., to model and simulate change) and future research. GMB fostered trust and idea sharing between participants.

Conclusion

GMB can facilitate learning about which outcomes are (or are not) impacted by interventions, which resources and approaches are required for quality implementation (e.g., implementation strategies), and tradeoffs in outcomes and resources between interventions. GMB also provides a structured, effective process to generate a shared implementation vision amongst participants. Lessons learned include methods for developing trust with and between participants, and the need for researchers to tailor GMB actions for participant and project needs.

Supplementary Information

The online version contains supplementary material available at 10.1186/s43058-024-00660-2.

Contributions to the literature.

• Group model building (GMB) is a promising method to support implementation activities throughout implementation phases, and to concurrently develop research products that contribute generalizable knowledge.

• GMB provides a systematic, transparent method to operationalize strategies such as conducting local consensus discussions, modeling and simulating change, and building a coalition.

• Engaged systems science methods such as GMB can develop strong research-practice partnerships and dynamic hypotheses of change, resulting in implementation plans that are more likely to achieve desired health impacts.

• Specifying complex, multi-faceted implementation strategies can highlight common threads amongst strategy goals to identify strategy mechanisms and increase implementation efficiency.

Background

Substantial knowledge has been generated about implementation determinants (i.e., barriers and facilitators) [1] and strategies [2, 3]; however, substantial challenges remain for understanding the complex relationships between implementation determinants and health outcomes to design implementation strategies [4, 5]. Rigorous participatory methods — particularly those that leverage systems thinking to account for complex determinant relationships — can provide insights critical to understanding health and social care quality gaps, identifying potential interventions and implementation strategies to close those gaps, and anticipating intervention and strategy impacts on implementation and health outcomes over time [6].

Group model building (GMB) uses systems science thinking and methods for engaging implementation actors and other community partners to foster shared understanding about problems, generate innovative solutions, and implement solutions [7]. It has been identified as a structured approach for selecting and tailoring implementation strategies given the system’s complexity (i.e., implementation context) [4] and for testing implementation strategy mechanisms [8, 9]. More fundamentally, GMB can be leveraged as an implementation approach — a structured method for reproducibly operationalizing implementation strategies and implementation processes (e.g., intervention adoption). GMB is particularly well-suited for operationalizing strategies that target collaboration and decision-making — phenomena in each implementation phase.

This paper will: a) briefly overview GMB; b) describe how implementation strategy dimensions can be operationalized with GMB; c) detail a case study wherein GMB was used to operationalize three common implementation strategies (conduct local consensus discussions, model and simulate change, and build a coalition) to support intervention selection; and d) propose next steps for testing GMB as an implementation approach across implementation phases.

GMB: Brief overview

GMB was developed to increase consulting clients’ acceptability of system dynamics models and implementation of modeling insights [7, 10]. It has been conducted with diverse groups of community members, clinicians, and healthcare leaders [11].

GMB primarily employs system dynamics [12], although recent studies applied other systems-based modeling methods such as agent-based modeling [13]. A distinguishing feature of system dynamics is its focus on characterizing feedback dynamics that drive behaviors over time (e.g., non-linear effects, delays, accumulations). Understanding these dynamics is critical for identifying the most effective intervention targets and anticipating implementation outcomes. GMB combines qualitative and/or quantitative modeling with semi-structured group activities or “scripts.” GMB aims align well with addressing critical implementation challenges. For example, without shared understanding of a problem’s underlying dynamics, it is difficult to reach consensus on which intervention(s) are best suited to affect that problem [14–16]. Moreover, system dynamics models developed or validated through GMB can reduce the selection and implementation of “fixes that fail” [17] by accurately characterizing the system’s dynamics and allowing users to test assumptions about potential innovation and implementation strategy impacts. GMB and fields such as human-centered design [18] have shown that including end-users in model development is key to model validity and uptake.

GMB as an implementation approach

This section describes how GMB can operationalize implementation strategies across seven key dimensions outlined by Proctor, Powell, and McMillen [19]: actors, actions, action targets, dose, temporality, implementation outcomes, and justification (Table 1).

Table 1.

Overview of How Group Model Building (GMB) Can Be Specified as an Implementation Strategy

Specification component	Specification guidelines [19]	Group Model Building specification	Case study example
Actors	Identify who enacts the strategy (e.g., administrators, payers, providers, patients/consumers, advocates, etc.)	GMB actors consist of a modeling team and facilitation team. Individuals can serve multiple roles within and across teams. Scripts detail any script-specific roles and specify the actions to be taken by modeling and facilitator team actors. Modeling team- includes at least one person trained in system dynamics and one trained in GMB (can be the same person). Often in partnership with the facilitation team, the modeling team is responsible for designing sessions, including identifying which scripts will be used, specifying the length and number of sessions, and mapping out when each script will be applied. The modeling team is also responsible for building causal loop diagrams and/or system dynamics simulation models during or between sessions to reflect the insights shared by partners. Depending on project goals, additional model analysis may or may not be completed by the modeling team. Facilitation team- can include members of the modeling team. It is common (and advisable) to include at least one person from the group being engaged (i.e., those implementing the intervention or making implementation decisions) as a “community facilitator.” The facilitation team is responsible for leading sessions by delivering scripts (see Actions), ensuring that sessions meet project goals, fostering divergent and convergent thinking, providing structures and strategies to mitigate any conflict, supporting communication with shared language amongst participants and between participants and the modeling team, and generally supporting dialogue. Facilitators do not have to be trained in system dynamics. Of note, GMB participants will vary by project goals and phase. For example, during exploration and preparation, GMB partners should be sufficiently familiar with the system of factors shaping focal health outcomes, as well as the delivery system (e.g., hospital, clinic, social service agency) in which intervention implementation will take place. This familiarity increases the likelihood that GMB will identify the full range of factors likely to impact implementation and health outcomes. While designing implementation processes and selecting implementation strategies during preparation, at a minimum, GMB should involve individuals with the capacity or power to make the implementation decisions being supported.	Modeling and facilitation team comprised of three doctoral-level researchers trained in public health system dynamics and decision science
Action(s)	Use active verb statements to specify the specific actions, steps, or processes that need to be enacted	Conduct interactive sessions with a group of decision makers, implementers, community members, and other key informants as appropriate to understand a problem, simulate change, identify and select action plans via consensus, and learn about the system affecting the outcome(s) of interest. Structure sessions with scripts, or activities, that have been tested and validated or demonstrated as promising in the group model building literature [7, 20]. A compendium of evidence-based and emerging scripts exists [21]. Scripts include session goals, input (e.g., models, other scripts), output (e.g., models, insights), materials, actor roles, language for facilitators to say aloud to introduce an activity or systems thinking concept, discussion prompts, and, at times, evaluation criteria (i.e., fidelity monitoring). Create script maps to specify how scripts align with project goals and to plan sessions (e.g., which scripts delivered when).	GMB scripts (e.g., “Concept Models”; “Variable Elicitation”) were tailored to the complex problem of child maltreatment through using relevant examples and discussion prompts (e.g., ask participants to define child maltreatment prevention). Scripts were also operationalized to define facilitation roles given study resources and team members. Scripts were assigned to one individual session and three group sessions, as outlined in Additional File 2
Action targets	Specify targets according to conceptual models of Implementation Identify unit of analysis for measuring implementation outcomes	Mental model change is a key action target across GMB applications [22, 23]. Mental models include participants’ understandings of problem determinants and potential actions to address that problem. Sustained mental model change is hypothesized to arise through participant learning [22] and be a pre-condition [24] for proximal and distal individual, group, and organization-level outcomes such as the acceptability of GMB and the action ideas generated through GMB, changes in group thinking quality and nature, and quality implementation of action ideas. Scott, Cavana, and Cameron [22] detail additional mechanisms through which mental model changes occur and propose proximal outcomes indicating mental model change, such as enduring changes in individual decision preferences and group cohesion. For implementation efforts, in particular, action targets could include the following individual and group-level targets: • decision-makers’ knowledge about their own personal preferences and values, as well as those of their peers/collaborators, that might shape which evidence-based interventions are acceptable and which might serve as implementation barriers or facilitators [22, 23, 25] • decision-makers’ knowledge about a health problem’s determinants and which interventions might be best suited to address these determinants and avoid unintended consequences [17, 26] • decision-makers’ satisfaction with the decision process, and in turn, commitment to implementation [22, 27, 28] • group-level alignment on and commitment to decisions, which in turn is hypothesized to impact implementation quality through the generation of implementation champions, enhanced resource allocation, enhanced motivation to implement, and other mechanisms [29, 30]	• Clear project vision, including synthesis of group values Products: Project Vision Statement; Group definitions of problem to be solved (child maltreatment) • ERIC strategies operationalized (GMB Script): Identify Local Needs (“Creating a Shared Vision”) • Deepened, systems-informed dynamic theory of child maltreatment determinants and theory of change for local child maltreatment prevention efforts based on addressing these determinants Product: synthesized causal loop diagram • ERIC strategies operationalized (GMB Scripts): Identify Local Needs, Model and Simulate Change; Conduct Local Consensus Discussion (“Hopes and Fears”; “Variable Elicitation”) • Group-level alignment on intervention selection decision Product: prioritized, consensus-driven list of three evidence-based child maltreatment prevention programs to adopt; causal loop diagram indicating which child maltreatment determinants were addressed by each selected prevention program • ERIC strategies operationalized (GMB Scripts): Conduct Local Consensus Discussion (“Dots”; “Initial Policy Options”; “Action Ideas”)
Dose	Specify dosage of implementation strategy	Varies by implementation stage, strength of partnership with actors, and implementation goals. Typically, at least 2 sessions lasting 90 min, each on separate days or with independent time between two sessions on the same day [7, 31]. Projects that aim to develop qualitative models or simulation models based on a common system dynamics model for which a validated quantitative model already exists will likely be shorter in duration than projects that aim to develop novel simulation models.	Four sessions conducted over approximately nine months One individual session ~ 90 min Three group sessions ~ 90–120 min See Additional File 2 for details on time allocated to each GMB script in each session
Temporality	Specify when the strategy is used	GMB could be applied at any implementation stage. However, it is likely most useful during the earliest stages of implementation (e.g., exploration and preparation). Example motivations for GMB within the phases of the Exploration, Preparation, Implementation, and Sustainment (EPIS) framework [32] include, but are not limited to: Exploration – When health service or status problem to be addressed are being identified and clarified, local needs are being assessed, system-level changes needed to improve problems are identified, and evidence-informed and/or innovative solutions are being identified and considered for adoption or actively adopted Preparation – When gaining support to implement the adopted interventions or innovation(s), generating an implementation plan (i.e., identifying implementation strategies) to address implementation barriers and facilitators Implementation – When addressing new barriers or facilitators that might require adapting the implementation plan, planning for sustainment, adjusting the implementation plan to obtain better implementation, service, and client outcomes, and generating support for sustained commitment to implementation Sustainment – When identifying needed system adaptations to facilitate sustainment, generating support for sustained commitment to implementation, disseminating results	Exploration – identifying local needs and strengths through feedback loops describing the dynamics shaping child maltreatment incidents and prevalence, identifying and prioritizing solutions (e.g., evidence-based prevention practices and policies) and needed system-level interventions that address interconnected child maltreatment determinants that are not fully addressed by current evidence-based practices and policies (EBPs)
Implementation Outcomes Addressed	Identify and measure the implementation outcome(s) likely to be affected by each strategy	• Adoption of evidence or specific innovation (given timelier decision-making and increased likelihood of consensus to adopt [29, 33, 34] • Acceptability of adopted evidence or innovation (given greater mental model alignment and perceived alignment of adopted interventions with mental models [22, 25, 35]); • Perceived, actual fit or appropriateness of the adopted evidence or innovation for the context in which it will be implemented (given more careful consideration of what the adopted innovation is intended to target and identification of intervention adaptations that might be required to improve fit) • Fidelity to implementation plan and fidelity to delivery of the innovation being implement, based on the hypothesis that if interventions are adopted with better fit and perceived acceptability, there will be less fidelity-inconsistent adaptations [36] • Reach/penetration [37] of the innovations being implemented (given greater commitment to implementation thus greater resource allocation to support reach [38]) • Sustainability of innovations adopted with or without GMB (through greater implementation commitment from engaged partners [35, 39] and careful planning for adaptations to the innovation or system to support sustainment [40])	Adoption, Acceptability: Prioritized list of three EBPs to adopt (from initial set of 7), derived through a set of GMB scripts — Initial Policy Options; Action Ideas; Dots — and an iteratively refined causal loop diagram of child maltreatment determinants (including intervention outcomes and mechanisms to affect those outcomes)
Theoretical Justification	Provide empirical, theoretical, or pragmatic justification for the choice of implementation strategies	A range of theories, frameworks, and principles from systems science, system dynamics, economics, decision science, social choice theory, implementation science, and community engagement can be drawn from to motivate the use of GMB for empowering individuals to collectively determine which solutions are best for their context (See Rouwette et al. [39] and Berard [41]). Example rationale includes: Bounded rationality [42], put forth in the economics and decision science literature and later integrated into the system dynamics literature, posits that mental models, or individuals’ views of how the world works, have diverse ways of defining what contributes to problem and what a successful outcome will look like. These conceptual boundaries often fail to account for all potential policy impacts. Actors often try to act on the system shaping a problem in ways that negate the actions of others trying to affect change. This uncoordinated, competitive push and pull leads to policy resistant and “fixes that fail” [17]. GMB hypothesizes that mental model changes, including the alignment of mental models within the group, helps create more expansive boundaries and alignment of decision-maker's boundaries so that their independent decisions are less likely to have competing goals. Systems thinking: Stave and Hopper [43] propose a taxonomy of seven dimensions along the systems thinking continuum: recognizing interconnections, identifying feedback, understanding dynamic behavior, differentiating types of flows and variables, using conceptual models, creating simulation models, testing policies. GMB scripts explicitly attend to one or more components, though work is still being done to specify which scripts are most effective to promote systems thinking competency. Community engagement and participatory research principles put forth that true empowerment of individuals to solve community problems comes from listening to the collective wisdom and lived experience of individuals in the communities affected by the “problem” and attempting to address the problem [44–46]. Collective learning and action are facilitated by listening and amplifying this wisdom. Successful academic-research partnerships are characterized by social and group processes such as a shared vision, trust, dialogue and mutual learning, collaborative decision-making, alignment of decisions with the vision, and cooperation [47]. Hovmand [31], Zimmerman et al. [48], and others (e.g., [7, 49, 50]) expand on the influence of and practice of incorporating a participatory epistemology in GMB. Implementation science posits that decision makers are hesitant to adopt interventions that don’t have good perceived fit for their context (for example, see EPIS [51] and Promoting Action on Research Implementation in Health Services (PARIHS) frameworks [52]). GMB explicitly aims to facilitate decision-making with collaborative consensus [29]. Decision theory and social choice theory tells us that people inconsistently apply decision criteria when evaluating evidence based on the types of evidence available, as well as their personal values and expertise [53, 54]. Commitment to decision implementation is tied to alignment of the decision-makers’ evaluation of evidence and their satisfaction with the group process [55–57]. Social choice theory also has mathematically and experimentally demonstrated the challenges in aggregating individual choices in a manner that reflects the rights [58] and preferences, or votes, of each individual (see Condorcet’s paradox, Arrow’s theorem [59, 60]. Such insights point to the need for aligning decision makers’ mental models (i.e., understanding of a problem and potential impacts of available solutions) prior to voting so that the group-selected outcome more equitably and accurately represents each decision maker’s choice. Decision conferencing frameworks outline how different problem types, including complex problems, require communication support, information processing, and process structuring. GMB addresses each of these process support dimensions, as further detailed by Luna-Reyes et al. [61].	Child maltreatment is a complex problem and requires a systems lens to understand root risk and protective determinants and to develop comprehensive visions for change and intervention. Complexity is defined by the theory that “everything is connected to everything else” [17]. Interconnections between what public health practitioners often refer to as risk and protective factors can be characterized by feedback loops that drive change by providing balance or escalating behaviors [12]. Interconnectedness results in nonlinear or delayed effects/ dynamics that create surprising behavior in which the sum of piecewise effects is different from the whole [12, 17, 62] and which must be accounted for during implementation planning. Child maltreatment prevention EBPs address some, but not all determinants, making GMB well-suited to help identify which suite of interventions could complement other interventions already implemented in a community to address child maltreatment determinants Child maltreatment policy decisions about how to intervene are group decisions that could benefit from structured processes for communication and information processing such as GMB [61]. Following participatory research frameworks, listening to individuals involved in these decisions and who have lived experience working with families who are involved with child welfare is critical to holistically defining the system affecting child maltreatment risk and identifying solutions to improve the structures and processes to help families thrive.

Actors

Actors deliver implementation strategies [19]. GMB involves a core modeling team and facilitation team. One individual can fill multiple roles.

The facilitation team delivers GMB sessions. They promote curiosity and dialogue between participants by audibly synthesizing participants’ expertise and insights [63], and by pointing out dynamic phenomena as participants describe complex problems and potential solutions. Facilitators ensure that logistic needs are met (e.g., room set-up, agenda setting), engage partners, and clarify scientific and practical objectives. The facilitation team often includes 3–5 individuals, including, at a minimum, someone trained in systems thinking and/or system dynamics (“modeler facilitator”) [64]. It is helpful for a facilitation team member to have substantive expertise about the modeled problem(s) to elicit relevant insights from participants, and to help the modeling team incorporate participants' insights and relevant scientific evidence during model development. Hovmand’s Reflector Feedback script encourages the facilitator to reflect on how participants’ comments align with scientific evidence during GMB session closure [21].

To mitigate power imbalances, it is also recommended to include a “community facilitator”— someone who is part of the community from which GMB participants are drawn or who has an existing relationship with the community [64]. Community members might co-facilitate GMB sessions and/or be involved in session planning activities such as selecting and tailoring scripts to community norms, experiences, and priorities, or selecting locations that foster psychological comfort for participants. Table 1 provides guidance on identifying community facilitators and GMB participants.

The modeling team is responsible for GMB session design, often in partnership with the facilitation team [29, 64]. The modeling team includes at least one person trained in system dynamics (or the selected modeling approach) to translate participants’ insights into qualitative and/or quantitative system dynamics models and foster systems thinking. For example, the modeler might calibrate the model to available data (e.g., trends in county health assessments, national surveys), make structural model changes to reflect participants’ mental models, and run simulations during or between sessions.

Action(s)

Scripts outline the primary GMB actions (i.e., activities) [19]. Scripts can be tailored for group size, focal problem, current implementation phase, and partner preferences [20, 64]. Established scripts are generalizable across projects, but can be tailored to project goals [20, 65]. Additional File 1 identifies scripts that align with common implementation strategies.

GMB projects for implementation span some or all of the following (often non-linear) arc [66, 67]: 1) learning about systems thinking — a systematic, “dynamic way of thinking” that is more likely to accurately characterize the root causes of a problem compared to traditional linear thinking [17, 68]; 2) applying systems thinking to complex problem solving; 3) developing and using system dynamics models to identify leverage points that drive system behavior and should be targeted by interventions [69]; and 4) identifying, operationalizing, and selecting implementation strategies by modeling how they could improve implementation and health (i.e., specifying mechanisms) [70]. Steps 3 and 4 could be modified based on the focal implementation phase.

Action targets

GMB action targets are the behavioral and cognitive phenomena hypothesized to change within implementation partners during GMB that subsequently impact implementation quality [71]. GMB encourages individuals to make their mental models — personal interpretations about how the world works — explicit [7, 67, 72]. For example, GMB may improve leaders’ understanding of the focal problem(s) (individual-level), alignment of leaders’ mental models (individual and group-level), and implementation commitment (individual-level and group level). Improved mental model alignment and commitment, in turn, may foster timely intervention adoption, effective implementation planning, and sustained implementation [73–77]. Further, GMB aims to foster systems thinking — a potential mechanism for identifying and adopting appropriate, acceptable interventions.

Dose

The dose or length of GMB projects can be as short as a half-day workshop or extend to multi-hour sessions over longer periods (e.g., weeks, months) [7, 64]. Group size and diversity will inform dosage [7, 64], as will project goals and resources.

Temporality

The Exploration, Preparation, Implementation, and Sustainment (EPIS) implementation framework describes implementation as occurring across four phases, often non-linearly (i.e., activities occur simultaneously across phases, are delayed, or revisited) [32, 51]. GMB is particularly well-suited for supporting exploration and preparation activities — phases for which implementation strategies are not nailed down [51] — including defining problems, adopting interventions, planning for implementation given context (i.e., the system’s dynamics), and selecting, tailoring, or adapting implementation strategies [78]. While other methods are typically suited for selecting and operationalizing either interventions or implementation strategies, GMB can be used to complete both activities. For example, GMB-developed models can inform intervention selection and operationalization by explicating problem dynamics [69] and whether, or how, evidence-based practices (EBPs) target interconnected determinants and feedback loops — closed connections that shape system behavior through reinforcing (escalating or de-escalating) or balancing processes [12] — driving problem dynamics. Models can also explicate how implementation contexts (e.g., barriers and facilitators) may dynamically change over time, thereby informing implementation strategy operationalization, timing, and sequencing. Anticipating system changes is necessary for implementation, as some barriers may be addressed through one strategy, but changes in that barrier may trigger the need for another strategy. For example, workforce turnover is a common implementation challenge [79, 80]. Turnover affects how many individuals can be served and will have varied impacts over time as a new clinician gradually increases their workload, requires less supervision, and increases alignment with the organizational culture and climate [79, 80]. Quantitative simulations, particularly those calibrated with local data, can help decision makers anticipate how and when limited resources should be deployed to mitigate undesirable impacts of such changes. Alternative implementation strategy selection approaches typically rely on lists of barriers, facilitators, and structural supports that minimally account for these dynamics [81, 82]. Regardless of data specificity to context, GMB-developed simulations can identify the patterns of systemic change that may occur during implementation [83, 84].

During implementation and sustainment, models can help plan effective responses (i.e., new or modified implementation strategies) to unplanned events such as withdrawn funding, political push-back, or natural disasters. For example, in implementation, models could simulate whether unplanned changes in Medicaid eligibility would result in a clinical hiring need or whether current capacity could absorb newly eligible patients. In sustainment, modeling could compare impacts on financial solvency of system behaviors such as waitlist growth across alternative training strategies (e.g., onboard multiple clinicians at once versus a staggered approach). GMB-generated products, such as models and materials describing them, can also be used for dissemination.

Implementation outcomes

GMB is hypothesized to primarily affect implementation outcomes such as adoption [29, 33–35], appropriateness [35], feasibility [85, 86], reach [37, 38], and sustainment [44, 87]. However, few studies report long-term outcomes [39, 88]. A 2023 systematic review of 72 studies leveraging GMB for implementation [11] reported outcomes such as model confidence (which could affect model appropriateness, reach) and positive system changes. Most outcomes measured thus far may be best characterized as outcomes that are causally (e.g., mechanisms) or temporally preceding implementation outcomes (e.g., adoption, fidelity, acceptability).

Theoretical justification

Three primary theoretical streams underlie GMB: systems thinking, participatory or engaged research, and decision science theories and frameworks such as social choice theory [39, 61, 89, 90]. GMB uses system dynamics models (causal loop diagrams (CLDs) and simulations) and scripts to foster systems thinking. Models illustrate how interconnections between factors lead to emergent behaviors [91] over time that are not easily intuited without simulation. Therefore, GMB hypothesizes that simulation models function as “boundary objects” — visual representations of reality — to develop shared language and mental models, [6, 7, 22, 67]. Shared language, in turn, facilitates trust and agreement about actions through attitude alignment, as detailed in the theory of planned behavior [92]. GMB may overcome barriers to model uptake and action with model insights by teaching participants the underlying methods and theories, and by co-developing the model with participants [35, 93].

Similar to participatory research principles, GMB emphasizes the need to honor GMB participants’ expertise as legitimate and critical for model validation. Engaged perspectives elevate community and practice-based knowledge to an equal or higher status than scientific knowledge. Researchers must critically reflect on power dynamics that shape problems, solution adoption, and implementation processes [44, 64, 94–97]. Active listening and pausing while facilitating sessions can support such reflections. Community facilitators, as described above, can also help address power imbalances.

Social choice theory is the inquiry (and set of models, theories, and frameworks) of collective decision-making processes and outcomes [59]. Group decisions need support in information processing, communication facilitation, and process structuring [61, 98]. GMB is hypothesized to provide these supports to achieve timely, transparent decisions likely to achieve their intended impact [29, 61]. For example, social choice theorists demonstrated social and mathematical challenges in accurately representing individual preferences in a collective decision, such as elections [59, 60]. GMB uses voting procedures (e.g., “Dots” script) that can reduce response bias and statistical noise often observed when aggregating votes [59, 99]. More importantly, GMB aims to align mental models so that participants are more likely to reach a shared conclusion (i.e., voting for the same option), thereby minimizing the need for synthesizing incongruent votes. Further, GMB can foster learning about how to make implementation decisions aligned with local contexts, which can confer benefits beyond the initial GMB-supported effort.

Methods

Case study

A pilot study aimed to support decisions regarding child maltreatment prevention EBP adoption for local (e.g., community, county) implementation. GMB was identified as an appropriate methodology because system dynamics focuses on defining how behaviors, such as child maltreatment, are the result of self-reinforcing or balancing behaviors (feedback loops) and structural determinants [100]. It is critical to understand how these determinants are interconnected to anticipate intervention consequences (e.g., whether a singular intervention might be insufficient or cause iatrogenic effects). Child maltreatment prevention EBPs typically focus on individual-level determinants, such as parenting skills, yet rarely target structural factors (e.g., social determinants of health such as health care access, stable and purposeful employment) that influence how parents might apply those skills. For example, parents may experience greater challenges in applying evidence-based parenting approaches when they experience mental health distress (due to mental health care barriers) or have minimal time to practice new approaches due to a heavy work schedule (structural employment challenges). EBP appropriateness should thus be defined by both the evidence base and whether the EBP targets prevalent multi-level determinants (e.g., community strengths, needs). EBP acceptability increases when decision makers perceive the EBP to address their priorities in desirable ways. Thus, to address child maltreatment and prepare for successful implementation, it is critical to identify which contextual determinants (e.g., community strengths, needs) are addressed, or not, by each potential EBP. It was hypothesized that GMB would improve actual and perceived alignment (appropriateness and acceptability, respectively) of the adopted EBP with local strengths and needs [101], and provide an efficient method for partners to narrow down which EBPs to consider.

Table 1 describes case study activities. The text below focuses on high-level, practical aspects.

GMB team and participants

The core modeling team — two doctoral-level trained system dynamics methodologists (KHL, LF) and a system dynamics doctoral student (GC) — engaged eight participants from North Carolina (NC). Priority was given to recruiting participants from diverse locations within NC (e.g., multiple counties) who had practice-based experiences as clinicians or administrators at child and family-serving organizations, as these individuals were identified as having professional goals and responsibilities aligned with the pilot project’s goal to identify interventions for child maltreatment prevention aligned with context. Participants were not in a pre-existing decision-making group. Participants or their organization were compensated $350.

Participants (n = 8) included a group home/foster care hybrid model administrator, a children’s advocacy center director, a county non-profit administrator, social workers, a school support specialist, a certified EBP facilitator, and a non-profit director. Five participants were parents. They were mid-career professionals and primarily identified as female (n = 6). One identified as LGBTQ. Participants primarily identified as White Non-Hispanic; one identified as Black.

Planning GMB sessions

The modeling team used ~ 8 h to plan overall session flow and assign roles. Planning involved identifying the number of sessions, session frequency and length, activities, and goals (Additional File 2). Each session required approximately 3 additional planning hours to consider: 1) reasonable participant time expectations, 2) time the modeling team required to distill information and build models between sessions, and 3) desired implementation and research products. The modeling team scoped the project to preventing child maltreatment in NC; participants later defined child maltreatment and prevention (Additional File 4). Figure 1 depicts how research and implementation products were co-developed across sessions.

Fig. 1.

Individual and Group Session Output and Facilitation Team Preparation. Legend: Output from each group model building session is indicated in squares. Italicized text between sessions indicates facilitation (research) team actions to prepare for each session

Actions iterated between the GMB principles of divergence and convergence [7, 20]. Divergent activities focused on eliciting heterogeneous insights and expanding understanding of child maltreatment risk. Convergent activities synthesized participants’ insights, priorities, and decisions.

GMB sessions

Participants completed one individual session prior to three group sessions from June 2018 through February 2019. Individual sessions and Group Session 1 occurred in person. Group Sessions 2 and 3 occurred virtually to make participation less costly and more feasible (e.g., reduce travel time).

Individual sessions

First-order goals were to develop a shared problem definition (e.g., child maltreatment) and to make participants’ mental models explicit. Individual interviews, facilitated by the lead modeler (GC), provided participants opportunities to share their mental models without being influenced by peers who might have had different priorities, power, experiences, and expertise [102]. Interviews occurred at a place of the participant’s convenience (e.g., workplace). The modeler first reviewed fundamental systems science concepts, posed the research objectives, and asked partners to share their definitions for four key concepts: systems, child maltreatment, maltreatment prevention, child well-being. The modeler then guided participants to share their mental models using CLDs. A visual presentation (e.g., PowerPoint) introduced additional systems concepts such as aggregation and feedback loops. The modeler pointed out parallels to the participant’s CLD. The participant then created a second CLD focused on a subset of factors they wanted to prioritize from their first CLD. While creating this second, more detailed CLD, participants were prompted to provide stories behind factors’ interconnections and feedback loops. After the session, the modeler refined CLDs with phenomena that were discussed but not drawn. Interviews closed with participants sharing their values around child maltreatment prevention, and their hopes and fears for the project [21].

Group session 1

The first group session had three objectives: 1) introduce participants to one another, 2) establish project definitions of the four concepts explored during individual sessions (systems, child maltreatment, maltreatment prevention, child well-being) and a project vision, and 3) expand, refine, and correct (i.e., validate) a CLD that synthesized individuals’ CLDs. To prepare for objective two and distill a project vision statement (e.g., purpose), the lead modeler blended the individual definitions for the four key concepts and reviewed notes about participants’ values and hopes. To prepare for objective three, the modeling team created a “loop story table” [12] describing partial or complete loop behaviors (Additional File 3).

To increase a sense of shared purpose amongst participants, the first activity reviewed their blended definitions (Additional File 4), vision statement, and prevention framework (Additional File 5). Participants suggested minimal changes and reached consensus on revisions.

Next, the modeling team oriented participants to the synthesized CLD by highlighting four salient feedback loops from the individual sessions: parenting skills knowledge; child welfare system involvement; low pay employment and childcare; and trauma treatment availability. Participants had access to a poster-sized print of the CLD and a web-based version.1 The web-based version was strongly preferred due to the ability to isolate and zoom in on factors and connections.

The synthesized CLD served several implementation exploration and preparation functions. It 1) introduced participants to others’ mental models so that they could identify how their mental models might align or could be updated — a critical step for activating the mechanism of mental model change, 2) visualized a dynamic hypothesis for how child maltreatment is perpetuated in the local context and might be prevented through EBPs, and 3) highlighted structural factors (e.g., barriers and facilitators) that might impact EBP implementation and be addressed through implementation strategies (e.g., financial incentives, transportation infrastructures). Although the CLD included factors reflecting local contexts, generalizable variable names were used (e.g., public transportation, child welfare placement) to increase CLD transferability to other contexts.

The “Behavior Over Time” script [21] expanded the CLD. This script was tailored for the project by asking participants to draw graphs about an adverse (feared) trajectory and a positive (hoped) trajectory for how a child maltreatment risk factor of their choice (e.g., youth exposure to substances) might change in their context over a discrete time frame (e.g., years). This activity served the dual purpose of identifying factors to be targeted through an EBP and providing informal trend data against which a simulation model could be validated.

Group session 2

This session aimed to gather participant feedback on an initial simulation structure (Fig. 2) and to select EBPs to model. Translating the CLD into a structure of “stocks” and “flows” is a standard step when creating a simulation [12]. Stocks represent aggregated variables (e.g., individuals in treatment). Flows are the processes through which levels change (e.g., hospital discharge rate) [17]. During translation, incomplete feedback loops in the CLD or inaccuracies can become exacerbated and limit the simulation’s validity [12, 64]. EBPs had to be selected during this session to ensure that the simulation structure incorporated both the leverage points targeted by EBPs (e.g., parent–child interaction quality) and leverage points important to participants but not otherwise targeted by EBPs (e.g., family crisis support, transportation). Participants’ perceptions of which EBPs should be adopted were for a hypothetical decision. However, they faced similar decisions through their professional responsibilities.

Fig. 2.

Initial Stock and Flow Simulation Structure Presented in Group Session 2. Legend: Stocks are represented in boxes. Flows are represented by double-lined black arrows. Single-line arrows represent connections between ancillary variables and flows or stocks

To prepare for this session, the modeling team translated the synthesized CLD into a stock and flow structure. Key decisions to establish the structure included: which type(s) of maltreatment to focus on; the analysis unit (household, child, or community), and how to define child and family outcomes. The modeling team selected child neglect as the focal maltreatment type for the project’s available resources and timeline because: 1) all factors prioritized by partners could be incorporated; 2) factors associated with neglect are almost always associated with those related to physical and emotional abuse, thereby priming the model to incorporate additional types of maltreatment, and 3) neglect is the most prevalent type of child maltreatment [103–107]. Consistent with the goals agreed upon in Group Session 1, seven EBPs categorized by an EBP registry as primary prevention programs for neglect [108] were identified for prioritization. The modeling team prepared EBP descriptions (Additional File 6) and distributed them electronically one week before Group Session 2.

During the session, the modeling team first explained their rationale to focus on child neglect. Participants found this agreeable. Subsequently, modelers led discussion about the simulation structure and decision points, including which outcomes to measure with stocks. Participants asked clarifying questions, such as whether families with previous child welfare involvement were modeled separately. Key questions about the model’s plausibility and representation of real-world phenomena (i.e., structure assessment, parameter assessment [12] were posed to participants. For example: Were flows realistically impacting stocks? How should risk factor accumulation be measured? Which protective factors should be included? Participants’ questions and clarifications resulted in an updated CLD (Fig. 3), feedback loop stories (Table 2), and simulation structure. Participants’ engagement with the CLD and simulation model’s structure was critical to ensure model structural validity and boundary adequacy [12, 61], and to increase model acceptability to partners. Feedback loops were further validated with the scientific literature and content experts (PL, LS; Table 2). This step is not required, as participants’ expertise is already a type of model validation. Lastly, participants prioritized three EBPs to simulate using a three-step process (Additional File 2) [21].

Fig. 3.

Iterated Causal Loop Diagram of Child Maltreatment Determinants Informing System Dynamics Simulation Model. Legend: Arrows of the same color comprise a feedback loop. The colors correspond to the feedback loop label (e.g., R1, B1) with the same colored arrow around it. + signs indicate a positive (reinforcing, either virtuous and desirable or vicious and non-desirable) relationship, in which variables move in the same direction (i.e., An increase in variable 1 causes an increase in variable 2). Key reinforcing loops are indicated with an “R” inside an arrow.—signs indicate a negative (balancing) relationship, in which variables move in opposite directions (i.e., An increase in variable 1 causes a decrease in variable 2). Key balancing loops are indicated with a “B” inside an arrow. Italicized variables are repeated across the diagram for clarity

Table 2.

Loop Story Table— Key Feedback Loops and Corresponding Stories of Feedback Behaviors That Affect Child Maltreatment Risk

Feedback Loop (color)	Loop Type Variables	Loop Story	Examples of corresponding theory, framework, literature* *Some behaviors are most fully explained through clinical and practical experience, and do not solely rely upon scientific theory, frameworks, or evidence
Problematic child behavior and parenting practices	Reinforcing—Escalating: (R1) Problematic child behavior + → perceived problematic behavior - → parent stress + → positive parent–child interactions - → problematic child behavior	As children demonstrate problematic behavior, parents need appropriate skills to deal with this behavior. In the absence of these skills, parents employ ineffective or harsh disciplinary practices In response, the child is likely to have escalate their behavior in that moment and over time (i.e., their initial problematic behavior at the next occurrence is from a higher elevated state). This increasingly frequent or more severe problematic behavior further stresses or annoys the parent and reduce the frequency of positive parent–child interactions (For example, through parental withdrawal or harsh discipline). Even when parents’ stress is not directly expressed toward their children, children sense their parent’s stress and are more likely to “act out” or increase their problematic behavior to experience interaction with their parents, even if negative.	Coercion Interaction Cycle, Escalation Curve (within the Coercion Interaction Cycle) [109–111] Mirror Neuron Hypothesis [112, 113] Social Learning and other behaviorist theories [114, 115]
	Reinforcing—De-escalating: (R2) Positive parent–child interactions - → problematic child behavior - → positive parent–child interaction	As parents have more positive interactions with their children, such as increased occurrences of positive reinforcement of child behavior, children are less likely to experience emotional dysregulation. In turn, children are less likely to participate in problematic behaviors. Parents are then cued to continue providing positive parent–child interactions because the improved child behavior reduces their stress.
Parent trauma effects on SU, depression, and maltreatment following child behaviors	Reinforcing—Escalating: SU (R3), depression (R4), Parent trauma - → parent healthy behaviors, self-care, positive coping strategies + → parent substance misuse - → parent stress (short-term) - → basic material resources (long-term) + → children’s physical needs met - → likelihood child removed from home + → child removed from home + → parent trauma Additionally: Reinforcing—Escalating: (R5) Parent stress - → positive parent–child interactions + → Children’s socioemotional needs met - → likelihood child removed from home + → child removed from home + → parent trauma	As parents experience increased mental health symptoms and/or substance misuse, they often have less time and parenting skills to meet their children’s physical, socioemotional, and supervisory needs (due to time spent attending to their own mental health needs, lack of energy to attend to child, etc.).Reductions in positive coping behaviors and increases in parent mental health symptoms also increase the likelihood that parents are more easily frustrated by child behaviors and view the child’s behavior as problematic (even when it is typical for the child’s developmental stage). A child, in turn, will likely increase their problematic behavior as they perceive their parents’ frustration, initiating the feedback loops described in problematic child behavior and parenting practices. As children’s socioemotional needs are unmet due to lack of parental time and attention or negative parent–child interactions, the likelihood that the child is removed from the home increases, and the resulting trauma increases both the child’s and the parent’s trauma history, continuing the escalating cycle of maltreatment risk and bridging this risk across generations (i.e., intergenerational).	Literature on Adverse Childhood Experiences (ACEs) [116] Intergenerational trauma literature: [117–120] Literature on trauma following child removal from home [121–125] Attachment Theories [126–129]
Basic needs stress	Reinforcing—Escalating: (R6) Material resources - → parent stress + → parental substance misuse - → material resources	When basic needs are unmet (material resources such as safe, stable, hygienic housing; food; health care), parents are increasingly stressed about how they will meet their own and their children’s needs. To cope with the stress, parents might be inclined to initiate or escalate substance misuse. Funds spent on substances further reduce the available funds to meet material resources needs, escalating the feedback loop.	Hardship theory [130] Family Stress Model of Economic Insecurity [131] Material resource deprivation, Social Determinants of Health [132–134]
Parental stress and substance use	Reinforcing—Escalating: (R6) parent stress + → parental substance misuse - → material resources - → parental stress (long-term) Balancing: (B1) Parent stress + → parent substance misuse - → parent stress (short-term)	As their substance misuse increases, over time, parents are likely to experience more challenges in their daily life such as friction in their relationships or interruptions in meeting daily responsibilities such as attending work or school, which can reduce their access to basic material resources. Additionally, as parental substance misuse increases, parents are less likely to employ positive parenting practices or skills that lead to positive parent–child interactions. However, participants also told a story of a short-term balancing loops in which parents may experience short-term reductions in stress due to the effects of substances. Over the long-term, however, stress from negative parent–child interactions and life stressors such as basic material resources deprivation can increase a parent’s desire to use substances again in order to cope with the stress, leading back into the escalating reinforcing loop.	Epidemiological and intervention research on parental substance use risk factors and parenting skills of parents who misuse substances [135–137]
Parental stress and peer support	Reinforcing—Escalating: (R7) parent peer support - → parent stress + → parent peer support De-escalating: (R8) parent peer support + → parent developmental knowledge and skills - → perceived problematic child behavior - → positive parent–child interactions - → parent depression symptoms - → parent peer support	Parent peer support reduces parent stress by making parents feel validated in their parenting concerns or exhaustion. Peers can also increase parents’ parenting skills and knowledge, which can increase the likelihood of positive parent–child interactions through decreases in parent stress and perceptions of child problematic behavior. This reduction in stress and empowerment to employ positive parenting practices improves parents’ overall mental health and can make them more willing to continue engaging with peers and vice versa (facilitate peers’ willingness to engage with them). Participants primarily emphasized the virtuous or escalating reinforcing loop. The literature also describes a vicious, de-escalating reinforcing loop. Conversely to improved peer support over time, as parents experience more stress and depression symptoms, they are more likely to withdraw from peers or peers will withdraw due to less frequent positive interactions with the parent.	Stratton’s Conceptual Model of Parent Stress and Family Interactions [138] Family Stress Model [139–141]
Parent depression and child’s socioemotional needs (indirect)	Reinforcing—Escalating: (R9) parental depression - → positive parent child interactions + → socioemotional needs met - → problematic child behavior - → perceived problematic behavior - → parent stress - → positive parent child interaction - → parent depression symptoms	Parental depression can directly impact children’s mental health and well-being. Children perceive their parents’ depression and can feel more depressed themselves. For example, children might blame themselves for their parents’ depression. Children’s mental health status can mimic their parent’s mental health due to biological and environmental factors as well. The child’s depressive symptoms might cause the child to withdraw, further increasing parental depression as parents yearn for a strong bond with their children yet experience a diminishing relationship. Further, children might increase their problematic behavior due to their own depression symptoms. Either course reduces opportunities for positive parent child interactions, further increasing parents’ depression symptoms.	Bi-directional impact of child and parent mental health [142, 143] Behavioral theories and interventions [144–146]
Parental employment and childcare needs	Reinforcing- Escalating: (R10) Parent stable employment + → parent income + → access to quality childcare + → placement in quality childcare - → need for childcare - → parental employment Balancing (B2) Parent stable employment + → parent income - → childcare subsidy + → access to quality childcare + → placement in quality childcare - → need for child care - → parental employment Balancing- abbreviated (B3) Parent stable employment - → time to supervise child - → need for childcare - → parent stable employment	Parents’ time at work reduces their available time to supervise their child, thus increasing parents’ childcare needs. If childcare is available and accessible, then the childcare need gap closes and children’s supervisory needs are met, allowing parents to maintain their employment. Parents’ income can increase the quality of childcare that they can afford. Parents with lower income might be eligible for public assistance such as childcare subsidies, which provide access to high-quality childcare as well and provide a balancing loop to the vicious cycle of needing childcare to find and maintain employment, yet needing childcare to maintain employment. Should childcare not be available, parents either reduce supervision for children or have to cease employment to maintain supervision.	Relationship between parental employment, income level and child maltreatment risk [147–149]
Family income and basic needs deprivation	Reinforcing—Escalating: (R11) Material resources + → children’s physical needs met - → parent (basic needs) stress + → parent substance misuse - → material resources	As parents have less income, their ability to purchase the material resources required to meet their children’s physical needs decreases. As children’s physical needs are unmet, they might experience increased internalizing or externalizing symptoms to cope. Furthermore, children might be reported as potentially experiencing neglect due to the lack of resources to meet their basic needs. As their children’s well-being declines, parents might experience increasing depressive symptoms because they want what is best for their children. Increased depression, particularly in the absence of supports such as peer support, can increase parent stress as parents feel less able to cope with the stress of basic needs deprivation and unsupported to meet these needs. To cope with stress, parents might misuse substances (Basic needs stress loop), which can further contribute to material resource deprivation.	Family Stress Model of Economic Insecurity [131] Relationship between poverty and child maltreatment risk [150–153]

The identified references are meant to be illustrative but not comprehensive. All reinforcing loops could be either escalating (non-desirable) or de-escalating (desirable), but the trajectory most often described by participants is included. In some instances, stories for both trajectories are described

Group session 3

The primary goal was to obtain additional feedback on the simulation structure—akin to member checking in qualitative research [154]—in preparation for parameterizing the model to simulate the three EBPs’ impacts on target outcomes [155]. Prior to this session, the modeling team updated the CLD and simulation structure based on Group Session 2 feedback. The updated CLD was presented to demonstrate changes and receive corrections. The session primarily focused on the simulation structure, as it had been substantially changed in response to feedback about how best to track neglect-related outcomes. Participants were asked about parameters’ reference modes (i.e., baseline trends) such as parent peer-support and stress, as well as structural decisions such as how to incorporate parents’ trauma history and the impact of positive parent–child interactions — two key targets identified by participants in individual and group sessions. Due to the pilot project’s limited time, additional steps to validate and calibrate (ensure that the model reproduces observed trends) the stock and flow structure for a quantitative simulation were conducted without participants. Thus, these steps are not reported here.

Results

GMB as the action for “conduct local consensus discussions”

Participants agreed that several phenomena were crucial to understanding dynamic child maltreatment risk and thus which interventions might mitigate risks: 1) multi-level trauma, including intergenerational trauma and trauma experienced by providers who interact with families, 2) parent stress due to emotional stressors or basic needs deprivation, 3) availability of mental health treatment for parents and children, 4) parental substance misuse, and 5) parent social supports, especially peer and crisis support. This list of potential intervention targets helped researchers and participants narrow the potential EBP list to a smaller, more manageable set, demonstrating the value of GMB as a planning implementation strategy. One prioritized EBP was not initially considered because it was not considered primary prevention in the EBP registry. It was added due to participants’ strong preference for a program with a peer support component [156, 157].

GMB helped prioritize target factors and developed a boundary object (the CLD) that demonstrated why each factor would be critical for maltreatment prevention. A follow-up study created a brief video (linked in Additional File 2) describing the key feedback loops driving maltreatment risk and how each prioritized EBP did or did not target risk factors.

“Building a coalition” with GMB

The collaborative, consensus-driven GMB process fostered trust and relationship building amongst GMB participants, as indicated by how readily participants asked questions of the modeling team and one another. Participants also discussed potential collaborations during the in-person session.

“Model and simulate change” with GMB

Similar to previous community-engaged modeling projects, participants’ insights about the risk and protective factor interconnections improved dynamic hypothesis accuracy [49, 67]. For example, participants pointed out that parent stress and trauma can have both direct and indirect effects on child behavior, whereas the modeling team had only modeled the indirect pathway. None of the considered EBPs addressed all key feedback loops (Table 2). Instead, participants’ insights highlighted how child maltreatment risk is a complex phenomenon that will require complementary interventions across multiple child and family serving systems to sufficiently address the interdependent causes over time. Delineating the feedback loops that drive child maltreatment was thus critical to prioritize which EBPs were most appropriate and acceptable for the hypothetical implementation context.

Discussion

GMB is a promising implementation approach for operationalizing a single implementation strategy or achieving interdependent objectives of multiple implementation strategies. Just as Hawe and colleagues suggest that health interventions are events that cause changes in complex ecosystems (e.g., healthcare settings or communities) [158], implementation science might benefit from conceptualizing implementation strategies as events within a complex implementation process. These “events” are interconnected such that the whole (context) is greater than the sum of its parts. Implementation strategies dynamically change the implementation context. Therefore, implementation approaches must account for how the system adapts to previously delivered interventions, strategies, and contextual changes [159, 160]. Implementation scientists should partner with implementers to identify how strategies will address interconnected implementation determinants and effects that emerge in the implementation context as a function of implementation processes. Implementation scientists should not merely match implementation strategies to determinants, but also attend to the fundamental objectives of each strategy and how sequencing or co-delivering implementation strategies impacts a system of implementation determinants and emergent behaviors.

Systems science methods, such as GMB, are ideally positioned to support strategy and intervention selection in both implementation theory and practice. In the case study, the biggest leaps in participant learning during GMB come through interacting with systems models and applying systems thinking to understand root causes of observed behaviors — something others have found [35, 87]. For example, while drawing their CLD, one participant realized that they had not been considering the integral role of transportation in another implementation effort. They decided to bring it up during their next workgroup meeting. Whereas other methods and frameworks, notably Implementation Mapping [161] and Implementation Research Logic Model (IRLM, [162]) might match implementation strategies to static determinants, GMB can help identify which determinants dynamically drive implementation success over time, thereby streamlining resource allocation or informing adoption of both innovative and evidence-based strategies. This case study also illustrates how formally modelling dynamic processes can be instrumental for identifying necessary upstream interventions to address a complex problem such as child maltreatment (e.g., parental stress, workforce availability). By consulting the CLD, participants saw how innovative interventions must accompany EBPs to fully address parental stress, and how doing so could affect subsequent risk factors (e.g., parent–child interaction quality).

The case study and empirical literature [29, 39] demonstrate how GMB can effectively impact mechanisms — such as communication quality, trust between practitioners and decision-makers, and motivation to change behaviors — hypothesized to be common across implementation strategies [2, 24, 163, 164]. Strong relationships, social networks, and leadership are essential to implementation success [77, 88, 165]. Investing in methods such as GMB during exploration and preparation can promote trust and communication between implementation partners that will facilitate effective implementation and sustainment. Participants’ connections and relationship quality) can affect group dynamics in sessions, and thus which insights can be elicited. Thus, measuring social network changes and participant involvement throughout GMB could further inform how GMB can be used to “build a local coalition.”

Practical lessons for using GMB

Consistent with other collaborative simulation methods [163, 166–169] and GMB projects [88], flexible delivery of GMB and building participants’ trust were critical to the case study’s success. Individual sessions were valuable for researcher-participant rapport building. The modeler became attuned to participants’ logistical preferences. For example, some participants preferred sticky notes to create CLDs, whereas some preferred to draw directly on paper.

Diverse scripts enhanced engagement quality and model accuracy. The Behavior Over Time script [21] evoked stories, potential interventions, and factors not discussed in individual interviews or group review of the synthesized CLD. These included: adverse childhood experiences screening, preschool expulsion, children using illicit drugs, subsidized childcare, trauma-informed practices availability, and child welfare involvement timing and intensity.

Alternative meeting modalities, such as virtual sessions, were responsive to participants’ needs yet allowed for research goals to be met. Researcher-participant rapport established during the in-person sessions allowed the subsequent virtual sessions to be engaging. The modeling team employed three procedures to facilitate virtual delivery: explaining logic for the transition, using video conferencing with screen-share, and sharing documents beforehand. While the pilot study occurred before COVID-19, the pandemic encouraged the development of new tools and practices to effectively deliver GMB online [170, 171].

Limitations

Most GMB projects can and should begin with the community identifying the problem(s) they wish to solve. However, it is not uncommon for researchers to approach potential partners and propose GMB [23, 67, 172]. Since this project was researcher-initiated, some process insights may differ from those within established academic-practice partnerships. Yet, the quality of conversations and engagement among partners speaks to the strength of GMB for fostering collaborations and insights. Thus, GMB could also be a method to operationalize the implementation strategy to “develop academic partnerships [173].” Similarly, given that participants were not a decision unit, they did not need to reach decision consensus. Thus, this case study cannot address the time required to foster mental model alignment sufficiently for a group to reach consensus about intervention selection. Future studies should explicitly measure group-level mechanisms such as mental model alignment and consensus to understand GMB’s impact on implementation outcomes such as adoption. Finally, although participants had deep familiarity with parents’ complex needs and strengths, they did not have lived experience as parents involved with child welfare. Engaging parents who participated in parenting EBPs could help identify risk and protective factors not addressed through EBPs.

Future research

Future work could explore which contextual factors affect the appropriateness of GMB compared to other implementation strategy selection approaches. For example, Implementation Mapping offers a structured process for identifying and selecting implementation strategies [82]; however, there is less focus on mental model alignment and no complexity modeling [174].

There is a growing evidence base of GMB evaluations and related measures [11, 87, 88, 175, 176]. Research should evaluate GMB’s impact on other individual and group-level mechanisms such as systems thinking. Additional File 7 includes recommendations for reporting core GMB processes and project characteristics, modified for implementation scientists based upon recommendations by Rouwette et al. (2002) [39]. These recommendations could be systematically expanded with implementation science theories to further specify the potential impact of each GMB planning decision on hypothesized mechanisms and outcomes [177].

Future work can explore GMB appropriateness for operationalizing additional implementation strategies. For example, while “facilitation” enjoys a strong evidence base [178, 179], GMB might be well-suited to operationalize specific components (e.g., establishing a vision, identifying implementation determinants) using systems science. There are numerous parallels between the action targets of GMB and facilitation, including strong partnerships, problem identification, action planning, and priority setting [178]. GMB also shares similarities with the oilcloth implementation planning strategy, during which a facilitator guides learning and conversation using a boundary object [180].

Conclusion

Without rooting implementation planning in systems thinking, implementers are at risk for identifying “fixes that fail.” Moreover, assuming that individuals have a shared understanding of a problem and its solutions (i.e., mental models) will lead to challenges when implementing and sustaining innovations. As one partner noted: “lack of understanding…connects to everything.” By fostering deep understanding of the problem and potential responses using systems thinking, GMB can cultivate commitment to implementing fixes likely to succeed and avoid what one participant described as getting “distracted by the shiny.”

Abbreviations

NC

North Carolina

GMB

Group model building

EBP

Evidence-based practice

CLD

Causal loop diagrams

Authors’ contributions

GC: Conceptualization; Data curation; Formal analysis; Funding acquisition; Investigation; Methodology; Visualization; Writing—original draft. BJP: Supervision; Validation; Writing – draft, review, & editing. LF: Conceptualization; Formal analysis; Methodology; Writing—review & editing. PL: Supervision; Validation; Writing—review & editing. CHB: Supervision; Validation; Writing—review & editing. LS: Validation; Writing- review & editing. KHL: Conceptualization; Formal analysis; Methodology; Writing—review & editing.

Funding

(Cruden and Hassmiller Lich) NC TraCS UL1TR002489; (Cruden) Doris Duke Fellowship for Child Well-Being; K01MH128761 (Cruden).

Data availability

De-identified photographs of original GMB materials (causal loop diagrams, models, posters) available upon reasonable request from the corresponding author.

Declarations

Ethics approval and consent to participate

This study was approved by the University of North Carolina-Chapel Hill Institutional Review Board, Project 18–0659.

Consent for publication

GMB participants consented to de-identified publication of activities and outcomes.

Competing interests

Not applicable.