Participatory system dynamics in implementation science practice: a scoping review of methods, contexts and outcomes

Abstract

Background

This scoping review documents participatory system dynamics (PSD) applications in implementation science (IS) studies following a recent, increased integration of the two fields in the USA. It aims to illustrate ‘how’ and ‘why’ PSD modelling improves understanding of determinants of implementation outcomes for quality improvement.

Methods

We queried PubMed and PsycInfo for PSD, IS and their synonyms (community-based system dynamics or group model building, dissemination, quality improvement, translational research or knowledge translation). USA-based empirical studies were included when they described synchronous participatory activities to define a modelling problem over time. Studies were included when PSD was used as an implementation research method or implementation practice strategy. Fifty-eight studies on concept mapping were excluded, as were 65 intervention mapping studies. Nine articles remained after full-text review.

Results

Most studies (n=7) investigated PSD itself as the intervention for understanding an implementation problem or use it as a tool to understand how to implement an evidence-based practice (n=4) or select an IS strategy (n=3). Most articles were case studies, investigating feasibility and knowledge translation during the preparation phase.

Conclusions

We recommend that implementation research and practice clarify whether PSD is used as a method to uncover contextual determinants or strategy to do so. PSD has strong potential to use enhanced participant buy-in and problem definition to understand ‘how’ implementation strategies account for cyclical and temporal determinants. Greater alignment between PSD activities, participatory theory, implementation phases and outcomes is needed to strengthen evaluation of PSD’s applications in IS.

Registration

The protocol for this paper is listed in the UNC Libraries repository (https://cdr.lib.unc.edu/concern/scholarly_works/6682xg09r).

WHAT IS ALREADY KNOWN ON THIS TOPIC

• Participatory system dynamics (PSD) participation is known to create buy-in and shared problem definitions; known facilitators in quality improvement and implementation science (IS). Therefore, they may be effective approaches when combined.

WHAT THIS STUDY ADDS

• Our scoping review is the first to document PSD’s ability to uncover implementation processes; with its most frequent use as a strategy itself to understand temporal, contextual and cyclical components behind ‘how’ an implementation problem exists followed by prioritising strategy selection.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

• With clarified methods and standardised reporting, PSD may complement IS approaches as a practice-based quality improvement tool for understanding partner buy-in, knowledge translation and feasibility of a programme in context; helping practitioners address buy-in and refine problem scope for enhanced success of implementation strategies.

Introduction

To accelerate quality care delivery in the USA, implementation science (IS) needs innovative methods to uncover how affected individuals view underlying causes of complex problems.^{1 2} Implementation strategies and frameworks involve diverse community partners through shared knowledge, local consensus discussions and modelling change.^{3 4} However, these frameworks rarely measure ‘how’ strategies account for determinants changed over time, so they may miss how dynamic community partner perspectives impact implementation strategies.⁵ Therefore, quality improvement projects would benefit from using community-engaged approaches to model complex, time-delayed and dynamic implementation problems.⁶

Existing community-engaged methods have been recognised for their potential to address gaps in contextual and dynamic implementation problems. For example, participatory system dynamics (PSD), a systems-based approach, engages diverse community partners in coidentification of how complex healthcare systems are driven by inter-related, feedback loops, and dynamic factors influence a problem’s development.7,9 Rooted in system dynamics (SD), PSD uses the stages of (1) variable elicitation, (2) graphs-over-time and (3) causal loop diagrams, with some variation, to uncover the local beliefs, perspectives and contexts that impact evidence-based practice adoption.710,13 Each stage is completed synchronously with all partners resulting in a final SD model that captures factors often overlooked in other types of models—temporal feedback loops and interdependent, multilevel interactions.¹¹ The process of participating in PSD is known to create buy-in and shared problem definitions, known facilitators in IS, and may therefore be an effective approach to add temporal and cyclical determinants to our understanding of the contexts, or the ‘how’, behind which implementation strategies exist.5,711 14 15

Accordingly, the quality improvement field has recognised the potential of PSD and other systems science methods—such as SD, agent-based modelling and social network analyses—as tools to streamline evidence-based practice.⁵ Over half of the published papers on this intersection occurred in the last 5 years.¹⁶ A 2023 review on systems science and IS found PSD to be the prevailing systems science tool in implementation projects, it did not focus on or detail PSD methodologies themselves.¹⁷ Consequently, a 2024 review looked at group model building in public health programme implementation; highlighting its ability to build consensus among collaborators, a common IS goal, but pointed to inconsistent reporting of outcomes.¹⁶ However, the 2024 study did not review, quantify or align articles with formal IS concepts or language, the aims of our review, and under half of the studies cited in the 2023 and 2024 reviews included USA-based studies.^{8 16 17}

IS and SD have a shared aim of understanding contextual determinants and may have a strong potential when combined. Therefore, our IS-centred paper documented extent of PSD use across five domains: (1) study aims (‘why’ PSD is being used), (2) modelling team composition, (3) modelling activities (‘how’ PSD can elicit cyclical, dynamic contexts), (4) implementation stage and (5) problem being modelled. Findings reveal how this method is used to address implementation problems and answer a call in the field to integrate systems and IS understanding.^{5 6 18 19}

Materials and methods

This scoping review utilised the Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) and aligns with these recommended reporting requirements.^{14 20 21} The published protocol for this review is stored in the UNC Libraries repository and can be found at this link.

Information sources and search strategy

The initial search was conducted in June 2022 with a final, updated search in January 2025. We queried PubMed and PsychInfo (2000–2025) for articles containing PSD and IS terms and their synonyms (table 1).¹⁴ Because PSD may be used interchangeably in the IS literature, we included the misappropriated uses of ‘concept mapping’, ‘intervention mapping’ and ‘systems mapping’ (italicised in table 1). This was a validation check, as these are common methods in IS that are not systems science or PSD (eg, participatory but more linear in their approach). These articles were eliminated manually in the title and abstract review. IS is also a growing field, so we included synonyms in alignment with other IS reviews (table 1).^{7 14 22 23} Specifically, we included ‘quality improvement’, ‘translational research’ and dissemination terminologies as a validation check and removed based on context with intent to assess context in the title and abstract screening.

Table 1. Search terms, synonyms, and operationalised definitions used in scoping review.

Search terms	Description
“Participatory system dynamics” OR “Community-based system dynamics” OR “Group model building”	These terms capture the Participatory System Dynamics (PSD) literature
[systems mapping OR concept mapping OR intervention mapping]AND	These terms were for validation check of the articles
Implement* OR “implementation science” OR “improvement science” OR	These terms intend to capture the implementation science literature
[“quality improvement” OR “translational research” OR disseminat* OR “knowledge translation”]	These terms were for validation check of the articles
Operationalised definitions of methods, strategies, practice and outcome
Implementation outcomes 27 28
Acceptability	Perception that an intervention is satisfactory or agreeable and works as expected
Adoption	Initial decisions to employ and evidence-based practice (eg, uptake)
Appropriateness	Perceived fit of an innovation to address the problem of interest (eg, suitability, usefulness)
Feasibility	The extent to which a practice can be employed in a particular setting
Penetration	The incorporation of a practice within a specific setting (eg, reach)
Sustainability	The extent to which an implemented project is maintained or institutionalised
Exploration, Preparation, Implementation, and Sustainment (EPIS) stages and contexts 26
Exploration	Community partner assessment of or use of PSD to define the problem scope
Preparation	PSD use to elicit barriers and facilitators to—or develop a plan for—implementation
Implementation	PSD use in the process of an implementation problem
Sustainment	PSD use to scale-up implementation efforts
Inner Context	Represents what is happening within the organisation implementing the evidence-based practice (EBP) (eg, staffing, policies, procedures, culture and climate)
Outer context	Represents the larger, external factors that affect implementation (eg, federal or state policies, funding, inter-organisation relationships)

Study selection

We included peer-reviewed studies that used a PSD approach to address or elucidate an implementation challenge. We excluded non-English, non-USA and not peer-reviewed or empirical research (eg, commentary or literature review). This study excluded international studies to understand funding and political landscapes specific to the USA, where system-based methods have only recently been integrated relative to the robust, long-standing, international tradition of SD projects.^{24 25} This approach centres more emergent uses of the method within the field of IS, developed within the USA over the last 20 years.¹ We defined ‘participatory’ as synchronous, codesigned and discussed in tandem with others (eg, face-to-face, even if virtual).^{7 11} For example, several papers used ‘participatory’ language but had participants’ review a premade model asynchronously, alone or online. Articles employing IS but not PSD methods, or vice versa; were excluded in the full-text review. We included PSD synonyms, such as ‘group model-building’ and ‘community-based systems dynamics (CBSD)’, to compare their use in IS settings. Group model building participants build a qualitative causal loop diagram (CLD) to aid in problem scope and decision-making modelling. PSD shares this same purpose, but often produces a quantitative simulation model finalised outside by researchers while community-based SD blends both approaches by involving community partners in all model processes, iteratively over time.¹⁰

Resulting articles were screened through Covidence, a systematic review software, with duplicates removed. One author screened titles and abstracts (LC) and each full text article was reviewed by at least two authors (AK, NS and LC) with discrepancies resolved through discussion.

Data charting

The authors used a customised data extraction template using Covidence software (online supplemental file 1). We extracted relevant study characteristics (eg, design, duration, participant type and number) then IS and PSD concepts. We quantified the use of IS theory, the implementation stages of the Exploration, Preparation, Implementation and Sustainment (EPIS) framework, and Proctor’s Implementation Outcomes.426,28 Two authors reviewed each article to extract and consensus code the implementation outcomes, EPIS setting and phase and PSD elements (AK, LC, LZ, NS).

Data reporting

To document PSD use in IS projects, we will report (in order): study use of IS as a strategy or practice to evaluate IS outcomes (study aims), modelling team, modelling activities employed and EPIS stage inner or outer context (problem to be modelled). These are delineated with their definitions in table 2. There was key differentiation between articles that evaluated PSD as a strategy (eg, service outcomes) versus as an outcome of the evidence-based practice that PSD was used to understand (reported as IS outcomes).²⁸ These are conceptualised and included in the full list of operating definitions in table 1. Sometimes an article would use a definition synonym instead of one of the explicit IS outcomes (eg, ‘maintenance’ instead of ‘sustainability’ or ‘reach’ instead of ‘penetration’). Although infrequent, in these cases, interpretations were compared and finalised. We also reported on different PSD ‘roles’, the presence or absence of validated scripts, participant compensation and other PSD process-level methodologies.

Table 2. Participatory System Dynamics (PSD) and Implementation Science (IS) combined aims and outcomes.

	What were the study aims? Evaluating PSD as intervention (process)* vs evaluating the evidence-based practice implementation (practice) acceptability, adoption, appropriateness, cost, feasibility, fidelity, penetration and sustainment outcomes27 28	Who was the modelling team? Types of research expertise, partner, and facilitation team members. *More detailed view in table 4	When did modelling occur? Exploration, Preparation, Implementation, Sustainment Phase (EPIS26);	What was the modelling problem? Outer setting, inner setting, innovation factors, bridging factors (EPIS26);	What were the modelling activities? (Causal loop diagrams (CLDs), hopes and fears, graphs over time, connection circles, etc).
Brown et al33	Effectiveness* of PSD as a strategy	Maternal health equity, community-engaged research, systems science and community modelling (trained), and two graduate research assistants	Preparation, implementation	Inner	Causal loop diagrams, variable elicitation, graphs over time (GOTs), mental models, boundary objects
Calancie et al29 (MA)	Feasibility, sustainability, other: ‘knowledge’, ‘engagement’, ‘trust’, ‘accessibility’	Child obesity, group facilitation (trained), and community-based intervention methods	Implementation	Inner	Causal loop diagrams, variable elicitation, connection circles, hopes and fears, gallery walk, boundary objects, mental models
Calancie et al30 (OH)	Adoption, feasibility, fidelity, per definition: sustainability (‘maintenance’)	Researchers, system dynamics expert, county board staff member, and external consultant	Exploration, implementation	Inner	Causal loop diagrams, feedback loops, graphs over time, hopes and fears, connection circles, variable elicitation
Calancie et al31 (SC)	Feasibility, appropriateness, other: diffusion	Two community partners (trained in modelling), community-based system dynamics expert, and research team	Preparation	Outer	Causal loop diagrams, hopes and fears, graphs over time, connection circles, feedback loops, mental models
Cruden et al15	Adoption, appropriateness, acceptability	Child welfare content experts, trained system dynamics methodologists, doctoral students	Exploration, preparation	Outer	Causal loop diagrams, feedback loops, graphs over time, stock and flow, boundary objects, hopes and fears
Egbuonye et al32	Other: Knowledge change, ‘adaptability’, alignment and ‘buy-in’ among community partners	Human impact partners	Exploration	Inner and outer	Causal loop diagrams, feedback loops
Loyo et al34	Cost*, per definition: adoption (‘policy selection’)	Collaboration consisting of members from NIH, RTI, CDC, and Austin/Travis County HHS along with a veteran systems modeller (trained)	Exploration	Outer	Causal loop diagrams
Wang et al9	Feasibility	Facilitators, process coach, assistant modeller, and community research assistant	Preparation	Outer	Causal loop diagrams, mental models
Zimmerman et al6	Feasibility, per definition: penetration (‘reach’)	Multi-disciplinary behavioural health staff, managers, nurses, psychiatrists, administrators, social workers and psychologists, and group facilitator (trained)	Implementation	Inner	Feedback loops, mental models

CDC, Center for Disease Control and Prevention; HHS, Health and Human Services; NIH, National Institutes of Health; RTI, Research Triangle Institute.

Results

The final sample of 9 articles was derived from the keyword search that resulted in 463 articles (figure 1). After removing duplicates, the title and abstract screen excluded 209 studies due to exclusion criteria. In the full-text screen, 112 more articles were excluded due to intervention characteristics that did not qualify as IS or PSD or used synonyms. Notably, some excluded studies employed PSD without IS aims (n=40) and vice versa (n=51), or engaged community partners individually, but not in group settings or synchronously (eg, not participatory) (n=31). Full enumeration of excluded articles is listed in figure 1.

Figure 1. PRISMA-ScR study inclusion diagram. PSD, participatory system dynamics; PRISMA-ScR, Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews.

Study design

The vast majority were case studies (n=8), and one was a study protocol for a case study.⁹ All studies took place in the USA and about equitably distributed with respect to geography (table 3). The majority (n=5) of studies were funded through private grants,929,32 the remaining were either funded through institutional-based innovation grants (eg, university innovation or pilot grants) (n=3),^{6 15 33} or through a think tank and city partnership (n=1),³⁴ with no studies funded through National Institutes of Health mechanisms.

Table 3. Study design, aims, and outcomes of included studies.

Author	Title	Study aims	Study design/location	Evaluation of study aims (qualitative, quantitative and mixed methods)	Clinical outcomes
Brown et al33	Planning, Implementing and Evaluating an Online Group-Model-Building Workshop During the COVID-19 Pandemic: Celebrating Successes and Learning from Shortcomings	The study aimed to (i) ‘describe the use and effectiveness of software used’, (ii) describe applied facilitation and engagement strategies and (iii) add to the literature on evaluation of online group model building (GMB) workshops investigating maternal mortality disparities employed during the COVID-19 pandemic.	Case study, Southwest	Quantitative evaluation and qualitative model Pre-/postevaluation of workshop effectiveness (based on 3 Kirkpatrick evaluation framework questions on reaction, learning, and behaviour). Rating of overall quality and feedback for improvements. Three post-session check-in meetings to refine the accuracy of the CLD	Self-reported behavioural and knowledge outcomes of participants
Calancie et al30 (OH)	Implementing and Evaluating a Stakeholder-Driven Community Diffusion-Informed Early Childhood Intervention to Prevent Obesity, Cuyahoga County, Ohio, 2018–2020	The study aimed to 1) implement the systems model as an intervention, 2) assess participant perspectives after taking part in the stakeholder-driven community diffusion (SDCD) approach and 3) assess the groups chosen strategies in alignment with the systems map	Case study, Midwest	Mixed methods evaluation and qualitative model Meeting summaries and model artefacts to measure implementation Online survey and pre-/postqualitative interviews to measure perspective change, Follow-up survey to measure action steps	Perspective shifts, knowledge, engagement, trust, campaign communication plan
Calancie et al29 (MA)	Implementing Group Model Building with the Shape Up Under 5 Community Committee Working to Prevent Early Childhood Obesity in Somerville, Massachusetts	The study aimed to 1) describe use of GMB with the project committee, 2) measure the effects on knowledge and perspective of the participants and 3) describe the developed health messaging project	Case study, Northeast	Mixed methods evaluation (for perspective change) and qualitative model Knowledge, engagement and trust were measured from elements of the Childhood Obesity Modelling for Prevention and Community Transformation (COMPACT) community-diffusion survey—three scale rating after each session Perspective change measured in nine survey and semi-structured interviews	Childhood obesity. ‘perspective shifts’ of participants
Calancie et al31 (SC)	Implementing a Stakeholder-Driven Community Diffusion-Informed Intervention to Create Healthier, More Equitable Systems: a Community Case Study in Greenville County, South Carolina.	The study aimed to 1) describe implementation processes of the CBSD project 2) report how the model impacted understanding of county systems and 3) compare the intervention to extant obesity reduction projects	Case study, Southeast	Qualitative model, no evaluation Additional products: identified ‘action areas’. Ranked solutions (eg, action areas) following completion of the model	Childhood obesity. Group endorsement of prioritised action areas
Cruden et al15	Leveraging Group Model Building to Operationalize Implementation Strategies Across Implementation Phases: an Exemplar Related to Child Maltreatment Intervention Selection	This study 1) described implementation strategy operationalisation and GMB contexts, 2) demonstrated GMB as a method for contextualising three implementation strategies	Case study, Southeast	Qualitative simulation model, no evaluation Post-individual group sessions to refine the individually-develops and translated CLDs. Additional final product included three EBPs to simulate through the model	Dynamics of child maltreatment risk and leverage points for EBPs Implementation strategy selection and ‘prioritisation’ for EBP delivery
Egbuonye et al32	Bridging the Gap Between Transformative Practices and Traditional Public Health Approaches	The project sought to 1) examine underlying patterns in the system, 2) identify leverage points and 3) understand how the community could learn and adapt from systems change with the ultimate pursuit of a more equitable community.	Case study, Midwest	Qualitative evaluation and qualitative model The project measures these aims through 1) development of systems loops to understanding of underlying patterns and 2) selected points of leverage and 3) a final development of a systemic challenge statement They also measured these components through real-time feedback on process and mapping outputs every workshop through ongoing committee meetings.	Develop leverage points for future action on increase equitable access to care. Measure differing perspectives among partners that challenge collaborations
Loyo et al34	From Model to Action: Using a System Dynamics Model of Chronic Disease Risks to Align Community Action	This article aimed to capture how an extant SD model was used to align multiple community partners in the development of a comprehensive strategy through 1) perception change, 2) commitment to change and 3) selected policy options for reducing chronic diseases rates and costs.	Case Study, Southwest	Mixed methods evaluation and quantitative model Perception change toward the impact of the possible interventions was measured through a pre-/postsurvey. Commitment, influence and confidence in making changes was a post-likert scale question. Cost was assessed through simulation of each bundle and policy options were selected with this assessment+the simulation results in mind	Cardiovascular disease risk Endorsement of selected policy strategies
Wang et al9	Building Community Resilience to Prevent and Mitigate Community Impact of Gun Violence: Conceptual Framework and Intervention Design	The study hoped to motivate community members to participate in a community-based examination of gun violence. Specifically, the model aims to understand determinants of gun violence through 1) community perspectives and 2) calibration of the model with local population-based data	Case study, protocol, Northeast	Qualitative evaluation and quantitative model Research team will provide feedback on modelling process. Quantitative model will be calibrated, validated and compared with qualitative findings from earlier components of the study.	Reduced gun violence Identification of existing community assets that can serve as building blocks for community-led interventions
Zimmerman et al6	Participatory System Dynamics Modeling: Increasing Stakeholder Engagement and Precision to Improve Implementation Planning in Systems	The study sought to determine if two possible implementation plans may increase EBP reach by 1) accounting for system interdependencies, 2) increasing stakeholder buy-in and 3) optimising local capacity for EBP expansion. The study tested PSD participation and calibrated and simulated the model in order to examine two VA implementation projects	Case study, qualitative, West	Qualitative evaluation and quantitative model Qualitative survey on modelling goals and feedback, quantitative model made use of hypothesis testing and simulation	Identification of policy and procedural mechanisms underlying capacity for implementation of psychotherapy implementation. Selection of preferred implementation plan Time saved compared to standard kitchen sink EBP selection

CBSD, Community based system dynamics; CLD, causal loops diagram; EBP, evidence-based practice; GMB, Group model building; SD, system dynamics; VA, Veterans Health Administration.

Study aims

All articles aimed to build a participatory systems model, the majority of which were qualitative (n=6) with fewer quantitative, simulation models (n=3). The models focused on a variety of different health topics, ranging from childhood obesity and Black maternal health to psychotherapies for post-traumatic stress disorder.^{6 30 33} All primary aims and evaluation methods are shown in table 3

The majority of studies (n=7) investigated PSD itself as the intervention or strategy paired with evaluation outcomes.69 29,33 The remaining studies (n=3) used PSD as a decision-making tool in implementation strategy selection.^{6 15 34} Therefore, further references to ‘intervention’ mean PSD as the process and answer the ‘why’ of motivating factors for PSD use. Calancie et al described this well when defining PSD as an intervention that influences participants’ thinking, decision making, and group cohesion and then continuing to refer to the PSD process as the intervention throughout.³⁰ This is an important distinction for IS, as PSD is often employed to promote participant subject matter understanding; for example, in evaluating topical understanding (eg, maternal health equity) along with its impact on improved implementation outcomes.

In order to evaluate the PSD process, studies used an about equal distribution of quantitative (n=1), qualitative (n=3) or mixed-methods (n=3) procedures, with two studies not formally evaluating the process. Quantitative evaluations were often pre/postsurveys, looking at perspective change (n=2), effectiveness, buy-in or session quality.30,33 Qualitative evaluations included surveys and semistructured follow-up interviews to look at perspective change^{29 30} or identify action items and levers.^{31 32}

Duration

Studies implemented interventions of varying length, scope and sequence. Two interventions took place over the course of 1 day,^{33 34} several months^{6 15 32} or as long as 2 years.²⁹ Most of the sessions conducted over multiple days never exceeded a 1-hour session per day.^{6 30} Others did not describe duration.^{31 32}

Participants

Studies varied in size and participant type (table 3). Samples ranged from 8 to 76 participants. Some studies used existing county and city coalitions or public health departments to recruit participants and implement PSD processes within these structures.^{29 30 32} Others recruited participants from larger studies,³⁴ distinct healthcare departments⁶ or independent of other funded projects.^{9 25}

The PSD interventions engaged various community partners, including programme administrators (n=7), clinicians (n=5) and researchers (n=5). However, articles inconsistently described the included participants. When described in depth, participants were often referenced for their field or sector, but not specific role.^{30 32} However, this may have been for anonymity given the studies’ sample sizes. Furthermore, only two articles included or planned to include persons with lived experience—a core element of community-based methodologies.^{9 33} Brown et al recruited two participants with experience of life-threatening birthing complications, and Wang et al pledged to recruit from groups differentially impacted by gun violence in their protocol.^{9 33}

Methods

Reporting

Studies used mixed IS and PSD terminology, often interchangeably (table 4). Seven studies investigated PSD primarily as an ‘implementation’ tool, followed by additional uses for ‘dissemination’ (n=3) and ‘quality improvement’ (n=1). The articles also used differing terminology to reference PSD as an intervention. The most common synonym was SD (n=7) with a participatory component, but only one article specifically used ‘participatory SD’ phrasing (6). All articles referenced group model building (n=9), even the PSD article, and the third-most common was CBSD (n=2).

Table 4. Participatory modelling methods of included studies.

Author	PSD theory	Session setup	Sample size, setting and payment	Types of knowledge/expertise engaged (eg, community-advisory board):	Facilitators and session roles	Use/availability of validated scripts	PSD methods description
Brown et al33	Five Rs, theory of interpersonal relationships (TIR)	One full-day (8 hour) virtual workshop. Opened with five Rs activity, behaviour-over-time-graphs, variable selection, final causal loop diagram (CLD) and action ideas matrices to develop mutual understanding of factors impacting black maternal mortality.	9 $200 compensation, 8 hours	Partners: clinicians; patients; lived experience; researchers; other: policymaker, doula, midwife, non-profit programme directors Research: maternal health content experts, systems modelling researchers, and two graduate research assistants Session/workshop expertise: two trained facilitators, etc.	Yes, two (of five) trained. No roles described.	Yes, scriptapedia and Hovmand.11 Full script not available. Citations for scripts sections listed in results table.	Group model building, system dynamics, causal loops; other: variable elicitation, graphs over time (BOTGs), mental models, boundary objects
Calancie et al29 (MA)	Stakeholder-driven community diffusion (SDCD)	Seven, 1 hour meetings utilised GMB through hopes and fears, graphs over time, variable elicitation, connection circles and CLD initiation and elaboration, followed by participant interviews and a follow-up interview, evaluation of the action plan and perspective shifts via survey.	16 stipend to individual or organisation	Partners: administrators; other: extant committee on healthcare: ‘Professionals from early childhood education and care (n=5), parks and recreation (n=2), the local health department (n=2), healthcare (n=3), food assistance programmes (n=1), and the public schools (n=3).’ Researchers: child obesity, community-based research and facilitation.	Yes, but unspecified number (‘by the Social System Design Lab at Washington University in St Louis’) No roles described.	Yes, scriptapedia Full script not available.	Group model building, system dynamics, causal loops; other: variable elicitation, connection circles, hopes and fears, gallery walk, feasibility and impact grid, boundary objects, mental models
Calancie et al30 (OH)	Community coalition action theory, diffusion of innovations theory, social network theory, SDCD	Over the course of 2 years, existing Shape Up Under 5 Committee members participated in six GMB-specific sessions which covered hopes and fears, connection circles, feasibility impact grids, gallery walks, variable elicitation and CLDs.	12 stipend, 50 hours	Partners: administrators; county-level coalition members (early ages healthy stages) Research: researchers, system dynamics expert, county board staff member and external consultant	Not reported	Yes, scriptapedia and Hovmand.11 Full script not available. Guiding questions listed in results table.	Group model building; causal loops, other: feedback loops, graphs over time, hopes and fears, connection circles, variable elicitation
Calancie et al31 (SC)	SDCD	Six extant city coalition committee meetings, the team led a SCSD project led hopes and fears, graphs over time and connection circle activities to develop action areas.	19 stipend, 30–40 hours	Partners: clinicians; administrators; other: executive directors, ‘pastor, health director, community member or programme officer’ Researchers: two community partners, CBSD expert and research team (6)	Yes, two community partners trained by research team No roles described.	Yes, scriptapedia Full script referenced elsewhere, primarily based on Hovmand text.11	Group model building, system dynamics, community-based system dynamics, causal loops; other: ‘hopes and fears’, ‘graphs over time’ and ‘connection circles’, feedback loops, mental models
Cruden et al15	EPIS; no formal participatory theories, but power redistribution and member-checking referenced48 49	Over the course of 9 months, one individual session and four 2 hour hybrid in-person and virtual group sessions were held to establish project definitions and refine the ‘synthesised individuals’ CLDs’.	8 $350 compensation; four, 2 hour sessions	Partners: administrators, lived experience, other: centre directors, social workers, parents of children in the child welfare system Researchers; two trained doctoral-level researchers and a doctoral student	Yes, two trained systems dynamics experts and a certified facilitator Modeller and facilitator	Yes, scriptapedia including Hovmand ‘reflector feedback’ script.11 41 Full script, with theory and citations, and agenda available in online supplemental files	Group model building; system dynamics, causal loops, stock and flow, other: graphs over time, boundary objects, mental models, hopes and fears
Egbuonye et al32	Mobilising for action through planning and partnerships (MAPP) framework, ‘leverage hypothesis’	Month-long session with Black Hawk County Health Department; force-field analysis, small group discussions, and a final causal feedback loop session to elucidate factors enabling or inhibiting equity	76 Payment not mentioned	Partners: clinicians; administrators; researchers; other: healthcare, mental health, education, economic Research: human Impact Partners	Not reported	Full script not available. However, theory-driven components were listed.	System dynamics, causal loops, other: ‘systems mapping’, ‘feedback loops’, ‘systems narrative’
Loyo et al34	Not reported	A 1 day ‘action lab’ meeting. Community partners used an extant SD model to simulate the effect of six possible interventions on cardiovascular disease risk. Individual endorsed interventions based on expected impact in a pre-/post-test of the simulation to measure perception and commitment change.	56 Payment not mentioned	Partners: clinicians; researchers; other: action lab consisting of healthcare, non-profit, advocacy groups, businesses Research: collaboration consisting of members from NIH, RTI, CDC, and Austin/Travis Country HHS along with a veteran systems modeller	Yes, one trained modeller No roles described.	Not reported	System dynamics, causal loop,
Wang et al9	Building resilience to disasters; social network analysis; community resilience conceptual framework	Over 3 years the research group will use a multi-pronged methods to reduce rates of gun violence, 1) social network analyses, 2) qualitative interviews, 3) spatial analyses and 4) a community-based systems dynamics model	Not reported	Partners: administrators, lived experience, researchers; others: police, community leaders, educators, health professionals, researchers and neighbourhood residents. Research: facilitators, process coach, assistant modeller, and community research assistant	Not reported	Not reported	Group model building, system dynamics, community-based system dynamics, causal loop, other: mental models
Zimmerman et al6	Plan-Do-Study Act (PDSA); EPIS; CFIR	During 1 hour monthly sessions over the course of 5 months, the study team calibrated the PSD model calibration and its simulation (1–3) to measure two implementation plans impact on reach of EBPs for post-traumatic stress disorder and depression.	Six teams, unspecified number, workload credit through VA	Partners: Patient, providers and policymakers Multi-disciplinary behavioural health staff, managers, nurses, psychiatrists, administrators, social workers, and psychologists. Quality improvement: authors reported that these activities were for the purpose of healthcare quality improvement not research. This included one ‘champion’ provider from every team	Yes, one trained facilitator, but not all	Not reported	Group model building, other: PSD, feedback loops, mental models

CFIR, Consolidated Framework for Implementation Research; CLD, causal loops diagram; EPIS, Exploration, Preparation, Implementation and Sustainment; GMB, group model building; PSD, participatory system dynamics.

PSD elements

The articles consistently used validated scripts; however, stipends and trained facilitators reporting varied (table 4). Five studies explicitly mentioned Scriptapedia, a commonly-cited digital repository of PSD session scripts.1529,31 33 Notably, these were predominantly publications from one author. One other article constructed scripts based on a community-driven strategic planning framework.^{32 35} Notably, only the newer studies made the scripts publicly available.¹⁵

Six studies explicitly mentioned facilitator training, the source and number of which varied.^{6 15 30 31 33 34} While only one article mentioned training location as the social systems design lab in Washington University, St. Louis’, it did not specify the number of individuals. The remaining studies had one^{6 34} or two trained facilitators.^{15 31 33} However, even with only one trained member, Zimmerman et al⁶ described ‘…over the few months of this pilot, frontline staff with no prior SD training engaged in the process and reported that PSD was useful’.⁶ Six studies discussed stakeholder compensation; only two specified exact amounts^{15 33} while others mentioned stipends29,31 or work credits.⁶

The articles used a variety of core PSD methods to uncover cyclical, dynamics of implementation problems. Though all articles mentioned the development of ‘causal loop’ diagrams, others used differing standard components (table 2). Only two studies contained all stages of ‘hopes and fears’, ‘graphs over time’ and ‘causal feedback loops’.^{30 31} Some studies (n=2) changed one element, but kept additional stages.^{15 29 33} For example, one article used behaviour over time graphs and causal loop diagrams but replaced variable elicitation with a five-Rs opening activity.³³ Another completed variable elicitation and ‘hopes and fears’, but instead used ‘connection circles’ in lieu of graphs over time.²⁹ Four studies used only CLDs, often alongside other non-standard PSD participatory components.^{6 9 32 34} For example, Egbuonye et al³² added a force field analysis (eg, enabling or inhibiting factors toward equity).³² In summary, the most common-listed PSD components beyond CLDs were feedback loops (n=6), mental models (n=6), graphs over time (n=4), ‘hopes and fears’ (n=3), variable elicitation (n=3), connection circles (n=3) and ‘stock and flow’ (n=2).^{6 15}

IS phases

This review investigated implementation phases in alignment with the EPIS framework.²⁶ Most studies investigated PSD use in advancing the intervention’s evidence-based practice during the ‘preparation’ (n=5) stage, followed by ‘implementation’ (n=4) and ‘exploration’ (n=4). Sustainability was never a primary outcome, but was often discussed.69 29,31 Approximately half of the studies (n=5) looked at more than one stage (table 2).

With regards to implementation stage, Loyo et al³⁴ described their use of the model as a precursor to a simulation; groups then worked together to begin implementation planning…not part of the simulation model… [but] implementation of the intervention as a vital conversation that happened as an extension of the model.³⁴ Wang et al⁹ delineated preparation with their intent to identify existing community assets needed to implement their project while also referencing the community-based methodology was chosen to increase the likelihood of intervention sustainability.⁹ Egbuonye et al³² described their project as exploration in how well they could infuse adaptive strategies into practice.³²

The embedded ‘inner’ and ‘outer’ contexts are part of the EPIS framework; five and five articles employed PSD to address these, respectively (table 2). For example, projects studying ‘outer’ factors often did so to aid in the action or priority area selection; or, in the case of Cruden et al,¹⁵ implementation strategy selection. Calancie et al described this as helping participants prioritise action areas to influence those systems (eg, food insecurity, state-level advocacy efforts, etc).³¹ Additionally, Cruden et al mentioned this selection explicitly in the third aim of the finalised CLD to highlight structural factors… that might impact EBP implementation and be addressed through implementation strategies,¹⁵ whereas, ‘inner’ factors included measuring outcomes related to the inner workings of an organisation’s culture, staff or policies such as Calancie et al’s (2022-MA) changes in [committee member] perspectives related to early childhood obesity.²⁹

IS theory

No consistent pattern emerged in the theories employed for the development and implementation of the PSD as an intervention (table 4). However, two studies used Diffusion of Innovation Theory to frame participant knowledge and perception-shift outcomes.^{6 30} Each of Calancie et al’s articles utilised Stakeholder-Driven Community Diffusion theory (n=3).³⁶ Zimmerman et al⁶ and Cruden et al¹⁵ were the only authors to explicitly reference IS frameworks. Both referenced the EPIS framework,^{6 15} while the former also referenced Plan-Do-Study-Act and the Consolidated Framework for Implementation Research (CFIR).⁶

Outcomes

Clinical outcomes

All articles used PSD as a method to address implementation challenges for a variety of healthcare topics, most commonly for obesity reduction (n=3),29,31 followed by rates of chronic cardiovascular disease rates, gun violence, health inequities and clinical psychotherapy capacity building^{6 9 32 34} (table 3). Several studies had explicit meta-aims to investigate PSD’s ability to increase stakeholder perception, buy-in and strategy endorsement.615 29,31

Implementation and process outcomes

Eight primary implementation outcomes were measured (table 2), with feasibility (n=5) most frequently cited.²⁸ This was followed by adoption (n=3), penetration/reach (n=2), sustainability (n=2), appropriateness (n=2), fidelity (n=1) and cost (n=1) (table 2). Several descriptions of feasibility outcomes looked specifically at participant perceptions of intervention feasibility. For example, in Calancie et al (2022-MA) participants ranked potential intervention areas for feasibility and impact.²⁹ Wang et al⁹ described that interventions … considered feasible by community stakeholders and effective in the simulation [were] the basis of interventions they will test and implement.⁹ Meanwhile, Zimmerman et al used PSD to evaluate if the post-traumatic stress disorder intervention could retain the feasibility of scaling to multiple locations.⁶

Although three studies did not explicitly use implementation outcome terminology, they met the criteria for these definitions (eg, reach, sustainability and adoption) (table 1). These included Zimmerman et al’s measurement of ‘reach’ (eg, ‘penetration’), Calancie et al’s ‘maintenance’ of relationships with decision-makers (eg, sustainability) and Loyo’s ‘policy selection’ (eg, adoption).

Several studies measured secondary outcomes, such as stakeholder knowledge translation and perception shifts.^{29 30 33} For example, Calancie et al reported that the intervention influenced [stakeholder] knowledge and perception shifts of the problem of early childhood obesity as measured by pre/postsurveys.²⁹ Brown et al also demonstrated a change in participant self-reports of knowledge of systems science/systems thinking.³³

Discussion

Summary

Overall, IS and systems dynamics methods, such as PSD, share the aim of uncovering contextual determinants. PSD elicits the temporal and interdepent nature of these determinants to enhance our understanding of implementation strategy decision-making. This article presents methods and outcomes from a scoping review on PSD use in IS projects in the USA, in order to advance replicability and quality. Four major findings emerged. Most studies (n=7) aimed to investigate PSD itself as the intervention (eg, ‘why’ it was used), as a decision-making tool to understand how to best implement an evidence-based practice (n=4) or select an IS strategy (n=3). This is an important distinction in the field where there is a call to understand temporal and dynamic mechanisms (eg, the ‘how’) by which evidence-based practices are implemented. Second, PSD was primarily used to solve a preparation-stage challenge, but often without specific reference to IS.²⁸ This was most often a ‘feasibility’ case study (n=5, 56%) or to measure change in participant ‘knowledge’ (n=3, 33%) as a secondary modelling process outcome. Third, studies employed mixed fidelity of the three core PSD elements. Authors seemed to select PSD activities based on resource and time restraints of a given project; all projects used causal loop diagrams, only half used ‘hopes and fears’ and ‘graphs over time’, two other core elements. In-depth, replicable reporting of the PSD methodology was limited overall. While this mixed use is not inherently adverse, the field of IS should clarify which stages are most relevant for its purposes and describe them in detail for replicability. Fourth, there was no clear pattern of IS or participatory theories—important for a nascent merging of the fields.

Alignment with PSD and IS literature

Our findings document PSD’s ability to uncover implementation processes, with its most frequent use as a strategy itself to understand temporal, context-dependent and cyclical components of an implementation problem followed by prioritising strategy selection (eg, motivating factors for its use).⁶ This holds further relevance for quality improvement projects, whereby PSD can disaggregate processes that go into driving improvement through team science. Its relevance for IS objectives confirms a recent 2023 study on systems science and IS where PSD was one of the most commonly employed methods.¹⁷ PSD was used in similar implementation phases, but to measure different implementation outcomes, than other IS literature. The most cited EPIS phases in the studies matched the order of frequency of appearance in IS literature; implementation, then preparation and exploration, but not sustainability, despite some studies measuring ‘sustainability’ outcomes.^{29 30 33}

Although ‘acceptability’ dominates implementation outcomes in the literature (~52.1% of articles), only one article reported this,^{15 27} while ‘feasibility’ was referenced almost two times as often compared with the literature (n=5, 56% vs 37%).²⁸ This may indicate that PSD projects may be most useful in IS as a tool for understanding a programme’s use in new settings (eg, feasibility) or the ‘how’ implementation strategies account for determinants–also underscored in other papers.^{15 18 37} The disparate use of theories and frameworks across studies reflects a call in IS to better clarify their applications.³⁸

These results reinforce PSD’s use as a tool for knowledge translation.³⁹ Typical of PSD projects, it was often used methodologically to increase participant learning, perception and ‘buy-in’ as an implementation outcome.¹⁰ PSD’s strongest asset lies in its ability to develop buy-in and shared solutions among partners.^{33 40 41} These facets closely align with constructs from the individual and inner-level Consolidated Framework for Implementation Research (CFIR) domains that are also commonly used in participatory implementation projects (eg, interpersonal, readiness and communication).3842,45 Study applications of PSD along these domains and outcomes hold promise for its use in IS; interpersonal group dynamics are a contextual determinant of evidence-based practice.

However, PSD and IS descriptions were inconsistent. Only about half of the articles reported on participant compensation, facilitator training and session scripts. PSD and group model building (GMB) were often used simultaneously and interchangeably, despite clear distinctions in the field.¹⁰ This aligns other PSD reviews, which note substandard process and participant descriptions and a lack of formal facilitator training.^{16 40 42} Future studies should report these process-level components for greater replicability and to more directly account for ‘how’ PSD elicits determinants from participants through selected activities.^{46 47}

Additionally, most articles did not thoroughly explore participatory elements (theory, incentives, guiding approach, recruitment) beyond participant makeup, neglecting true participatory depth. For example, few articles indicated justification for, selection of, or grounding in community-based participatory literature theories, though some did mention ‘community coalition action theory’ and ‘theory of interpersonal relations’.^{30 33} These were also the only articles to adjust their methods for power and interpersonal dynamics inherent in group settings. Without a strong grounding in participatory tradition, PSD use risks perpetuating community buy-in and evidence-based service delivery delay problems that it seeks to solve. The emerging PSD use in IS necessitates clear instructional resources and consistent reporting to aid in replication efforts.

Limitations

This review did not include international studies despite their robust history in international systems science research.^{24 25} This may introduce selection bias; however, the exclusion was intentional, due to the origins of IS in the USA and the authors believe PSD to be uniquely positioned to uncover the political and funding constraints of domestic settings. Several articles from the same author (Calancie) may introduce authorial bias. As a scoping review, we did not complete quality assessments, meaning this review may have research of mixed quality. In fact, there was evidence of disparate fidelity of PSD (eg, participant descriptions, time frame, unit of implementation, stages, etc). Scoping reviews are broader in their inclusion, so we looked at IS and PSD even if they did not meet strict definitions. This can lead to methodological heterogeneity but can also challenge direct comparisons. Future studies should prioritise this standardisation and assess risk, bias and the international basis of PSD in IS.

Conclusion

Participatory SD may answer a call to use systems science methods to understand ‘how’ implementation strategies are or are not successful in certain, dynamic contexts by uncovering them directly from problem owners. Furthermore, it may complement IS approaches as a practice-based quality improvement tool for understanding partner buy-in, knowledge translation and programme context feasibility.³⁹ Implementation scientists can use PSD methods to address buy-in, define problem scope and determine how to apply an implementation strategy in new settings (eg, ‘why’ it is used). This can be enabled through more rigorous participatory methods, clarified implementation outcomes and improved reporting on theory, data collection, study design and model fidelity (see Cruden et al).¹⁵

Data availability statement

Data are available upon reasonable request.