Advancing the reporting of mechanisms in implementation science: A guideline for reporting mediation analyses (AGReMA)

Abstract

Well-conducted mediation analyses have the potential to move implementation science forward by better understanding how or why implementation strategies cause their effects on outcomes. The AGReMA statement provides authors with recommendations for reporting primary and secondary mediation analyses of randomized trials and observational studies. Improved reporting of studies that use mediation analyses could help produce publications that are complete, accurate, transparent, and reproducible.

Understanding the mechanism(s) through which implementation strategies achieve their effects on outcomes is critically important to refine and adapt interventions for increased effectiveness, scale-up, or translation from one setting to another (Lewis et al., 2021; Wolfenden et al., 2021). Without knowledge of the underlying mechanisms, it is difficult to learn from ineffective implementation strategies or to design new implementation strategies for different priority populations which may have unique circumstances and needs. Despite the importance of understanding how and why implementation strategies cause their effects on outcomes, the mechanisms of implementation strategies are rarely tested, and where testing has occurred, is often incomplete or inappropriate (Lewis et al., 2020; McIntyre et al., 2018).

To best understand the mechanisms of implementation strategies, rigorous quantitative and qualitative methods are required (Lewis et al., 2020; Moore et al., 2015). Mediation analysis is a quantitative tool which can be used to estimate the effects of implementation strategies that work through causal mechanisms (Vanderweele, 2015). Mediation analysis involves the investigation of intermediate variables or mediators, which may account for the relationship between the intervention or implementation strategy and the outcome of interest (Lewis et al., 2018). Through mediation analysis, the total effect of an implementation strategy on an outcome can be decomposed into an indirect effect, which operates through the selected mechanism of interest, and a direct effect which operates through all other mechanisms. For example, Lee et al. used mediation analysis to investigate the mechanisms of implementation strategies underlying the effect from three complex intervention trials compared to usual care on nutrition policy uptake in schools and childcare services (Lee et al., 2018). Using data from 121 Australian schools and childcare services, the authors found that none of the four hypothesized constructs from the Theoretical Domains Framework (knowledge, skills, professional role and identity, and environmental resources) mediated the effect and that most of the intervention effect was left unexplained.

The value of investigating the mechanisms of implementation strategies has been recognized by National funding organizations such as the UK Medical Research Council (Moore et al., 2015) and the US Agency for Healthcare Research and Quality (Lewis et al., 2021). In line with a call for increased investigation of implementation mechanisms, there is a need to reflect on reporting practices that underpin the communication of these findings. Problems with the reporting of mediation studies in health and medical research have been recently highlighted (Cashin et al., 2019; Vo et al., 2020) and motivated the development of A Guideline for Reporting Mediation Analyses (AGReMA) (Lee et al., 2021).

AGReMA is an evidence- and consensus-based reporting guideline developed to provide recommendations for studies reporting mediation analyses. Through this minimum set of recommendations, the AGReMA statement aims to improve the completeness, consistency, and accuracy in reporting of mediation analyses. The AGReMA statement was developed using the Enhancing Quality and Transparency of Health Research (EQUATOR) methodological framework for developing reporting guidelines (Cashin et al., 2020; Moher et al., 2010). AGReMA was strongly informed by empirical evidence and substantive methodological expertise to guide the investigation of mechanisms for health interventions and implementation science. Full details on the development process of AGReMA are given elsewhere (Lee et al., 2021). Briefly, the development process included an overview of systematic reviews to assess the need for a reporting guideline (Cashin et al., 2019); systematic review of relevant evidence on reporting mediation analyses (Vo et al., 2020); a Delphi study that rated the importance of proposed reporting items by panel members (methodologists, statisticians, clinical trialists, epidemiologists, psychologists, applied clinical researchers, clinicians, implementation scientists, evidence synthesis experts, representatives from the EQUATOR Network, and journal editors) (Cashin et al., 2021); a consensus meeting; and a 4-week external review and pilot test that included methodologists and potential users of AGReMA. The final AGReMA checklist can be found in Table 1 and is available at https://agrema-statement.org.

Table 1.

A guideline for reporting mediation analyses (AGReMA) checklist.

Section/Topic	Item number	Item description
Title and abstract
Title	1	Identify that the study uses mediation analysis
Abstract	2	Provide a structured summary of the objectives, methods, results, and conclusions specific to mediation analyses
Introduction
Background and rationale	3	Describe the study background and theoretical rationale for investigating the mechanisms of interest. Include supporting evidence or theoretical rationale for why the intervention or exposure might have a causal relationship with the proposed mediators. Include supporting evidence or theoretical rationale for why the mediators might have a causal relationship with the outcomes
Objectivesa	4	State the objectives of the study specific to the mechanisms of interest. The objectives should specify whether the study aims to test or estimate the mechanistic effects
Methods
Study registration	5	If applicable, provide references to any protocols or study registrations specific to the mediation analysis, and highlight any deviations from the planned protocol
Study design and source of data	6	Specify the design of the original study that was used in mediation analyses and where the details can be accessed, supported by a reference. If applicable, describe study design features that are relevant to mediation analyses
Participants	7	Describe the target population, eligibility criteria specific to mediation analyses, study locations, and study dates (start of participant enrolment and end of follow-up)
Sample size	8	State whether a sample size calculation was conducted for mediation analyses. If so, explain how it was calculated
Effects of interesta	9	Specify the effects of interest
Assumed causal model	10	Include a graphic representation of the assumed causal model including the exposure, mediator, outcome, and possible confounders
Causal assumptionsa	11	Specify assumptions about the causal model
Measurementa	12	Clearly describe the interventions or exposures, mediators, outcomes, confounders, and moderators that were used in the analyses. Specify how and when they were measured, the measurement properties, and whether blinded assessment was used
Measurement levels	13	If relevant, describe the levels at which the exposure, mediator, and outcome were measured
Statistical methodsa	14	Describe the statistical methods used to estimate the causal relationships of interest. This description should specify analytical strategies used to reduce confounding, model building procedures, justification for the inclusion or exclusion of possible interaction terms, modelling assumptions, and methods used to handle missing data. Provide a reference to the statistical software and package used
Sensitivity analyses	15	Describe any sensitivity analyses that were used to explore causal or statistical assumptions and the influence of missing data
Ethical approval	16	Name the institutional research board or ethics committee that approved the study. Provide a description of participant informed consent or ethics committee waiver of informed consent
Results
Participantsa	17	Describe baseline characteristics of participants included in mediation analyses. Report the total sample size and number of participants lost during follow-up or with missing data
Outcomes and estimatesa	18	Report point estimates and uncertainty estimates for the exposure-mediator and mediator-outcome relationships. If inference concerning the causal relationship of interest is considered feasible given the causal assumptions, report the point estimate and uncertainty estimate
Sensitivity parameters	19	Report the results from any sensitivity analyses used to assess robustness of the causal or statistical assumptions, and the influence of missing data
Discussion
Limitationsa	20	Discuss the limitations of the study including potential sources of bias
Interpretationa	21	Interpret the estimated effects considering the study's magnitude and uncertainty, plausibility of the causal assumptions, limitations, generalizability of the findings, and results from relevant studies
Implications	22	Discuss the implications of the overall results for clinical practice, policy, and science
Other information
Funding and role of sponsor	23	List all sources of funding or sponsorship for the mediation analysis and the role of the funders/sponsors in the conduct of the study, writing of the manuscript, and decision to submit for publication.
Conflicts of interest and financial disclosures	24	State any conflicts of interest and financial disclosures for all authors
Data and code	25	Authors are encouraged to provide a statement for sharing data and code for the mediation analysis

This checklist is copyrighted by the AGReMA group under the Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported (CC BY-NC-ND 3.0) license.

^a Items included in the AGReMA Short Form checklist.

The scope of AGReMA covers primary and secondary mediation analyses of randomized trials and observational studies, and it is intended to be general so that it can guide the reporting of most mediation analyses. Authors are encouraged to use the 25-item AGReMA Statement for studies in which mediation analysis is the primary focus of a paper, and a 9-item short form, AGReMA-SF (items 4, 9, 11, 12, 14, 17, 18, 20, and 21 of the full checklist), for studies in which mediation analysis is a secondary focus of a paper, for example when mediation analyses are reported as supplementary to the main randomized trial or observational study as is often the case in implementation science. However, we stress that where possible, authors should use the AGReMA long form (25-item) to improve completeness of reporting.

Implementation Research and Practice will soon join the ranks of journals that have adopted the guideline requiring authors to include a completed AGReMA checklist alongside all submitted mediation studies. When using the short form of AGReMA, it should be accompanied by a completed Consolidated Standards of Reporting Trials (CONSORT) checklist (Schulz et al., 2010) for randomized trials, the Transparent Reporting of Nonrandomized Evaluations (TREND) checklist (Des Jarlais et al., 2004) for non-randomized evaluations, or the Strengthening the Reporting of Observational studies in Epidemiology (STROBE) checklist (Von Elm et al., 2007) for observational studies to ensure all elements of the study are completely and transparently reported.

Authors are encouraged to consult the AGReMA statement early in the planning of their mediation studies to ensure that appropriate consideration is given to the relevant assumptions, challenges, and issues of interpretation inherent to mediation analysis before data collection. In this way, AGReMA may also help to improve study design, measurement (e.g., data collection and its timing), and the conduct of mediation analysis. The AGReMA statement and checklists can also guide peer reviewers and journal editors when assessing mediation studies for publication. However, it should be noted that AGReMA was not designed as tool to assess reporting quality or the risk of bias in mediation studies (Vo et al., 2022).