A Guideline for Reporting Mediation Analyses of Randomized Trials and Observational Studies

Abstract

IMPORTANCE

Mediation analyses of randomized trials and observational studies can generate evidence about the mechanisms by which interventions and exposures may influence health outcomes. Publications of mediation analyses are increasing, but the quality of their reporting is suboptimal.

OBJECTIVE

To develop international, consensus-based guidance for the reporting of mediation analyses of randomized trials and observational studies (A Guideline for Reporting Mediation Analyses; AGReMA).

DESIGN, SETTING, AND PARTICIPANTS

The AGReMA statement was developed using the Enhancing Quality and Transparency of Health Research (EQUATOR) methodological framework for developing reporting guidelines. The guideline development process included (1) an overview of systematic reviews to assess the need for a reporting guideline; (2) review of systematic reviews of relevant evidence on reporting mediation analyses; (3) conducting a Delphi survey with panel members that included methodologists, statisticians, clinical trialists, epidemiologists, psychologists, applied clinical researchers, clinicians, implementation scientists, evidence synthesis experts, representatives from the EQUATOR Network, and journal editors (n = 19; June-November 2019); (4) having a consensus meeting (n = 15; April 28-29, 2020); and (5) conducting a 4-week external review and pilot test that included methodologists and potential users of AGReMA (n = 21; November 2020).

RESULTS

A previously reported overview of 54 systematic reviews of mediation studies demonstrated the need for a reporting guideline. Thirty-three potential reporting items were identified from 3 systematic reviews of mediation studies. Over 3 rounds, the Delphi panelists ranked the importance of these items, provided 60 qualitative comments for item refinement and prioritization, and suggested new items for consideration. All items were reviewed during a 2-day consensus meeting and participants agreed on a 25-item AGReMA statement for studies in which mediation analyses are the primary focus and a 9-item short-form AGReMA statement for studies in which mediation analyses are a secondary focus. These checklists were externally reviewed and pilot tested by 21 expert methodologists and potential users, which led to minor adjustments and consolidation of the checklists.

CONCLUSIONS AND RELEVANCE

The AGReMA statement provides recommendations for reporting primary and secondary mediation analyses of randomized trials and observational studies. Improved reporting of studies that use mediation analyses could facilitate peer review and help produce publications that are complete, accurate, transparent, and reproducible.

Health interventions and exposures often work through biological, psychological, and social mechanisms. These mechanisms can be quantitatively evaluated using mediation analyses (an analytic method commonly used in medicine, epidemiology, psychology, and the social sciences).^1,2 The principal aim of mediation analyses is to estimate the extent to which an intervention or exposure may affect an outcome through a potential causal mechanism. The findings from mediation analyses can advance theory, inform policy, optimize interventions, and facilitate the implementation of policies and interventions to clinical and public health practice. The value of mediation analyses of randomized trials and observational studies has been recognized by national funding organizations such as the US National Institutes of Health and the UK National Institute for Health Research.^3,4 Most mediation analyses are reported within the primary publication of a randomized trial or observational study, or as a separate report with reference to the primary publication. Even though the number of such publications has increased since 2014,⁵ recent reviews have shown that reporting is varied and often incomplete.^6,7

The aim of this initiative was to develop an evidence- and consensus-based reporting guideline for studies reporting mediation analyses (A Guideline for Reporting Mediation Analyses; AGReMA). The AGReMA project aimed to produce a long and short form to support primary or secondary reports of mediation analyses. This Special Communication describes the methods that were used to develop the guideline, provides long- and short-form checklists to be used when writing research reports, presents brief explanations for each reporting item, and provides guidance on how to use AGReMA.

A glossary of terms used in this article and in the long- and short-form checklists appears in the Box. Terms such as direct effect, indirect effect, and path-specific effects are conventional terminology for mediation analyses because the purpose of these analyses is to test hypotheses about potential causal effects. However, caution is warranted in interpreting estimated effects as causal inferences because causal assumptions (ie, there was sufficient control for mediator-outcome confounding) may be unmet, even in the context of a randomized trial of a treatment.

Box. Glossary of Conventional Terms Used in Mediation Analyses.

Action theory:

A theory that supports the hypothesized relationship between an intervention or an exposure and a given mediator.

Collider:

In the context of mediation analyses, a collider is a variable that is caused by the intervention or exposure and mediator, by the intervention or exposure and outcome, or by the mediator and outcome. Conditioning on a collider by design or analysis may induce selection bias.

Conceptual theory:

A theory that supports the hypothesized relationship between a mediator and a given outcome.

Confounder:

In the context of mediation analyses, a confounder is a variable that causes the intervention or exposure and mediator, the intervention or exposure and outcome, or the mediator and outcome. Uncontrolled confounders can induce confounding bias.

Consensus panel:

A group of experts representing relevant methodologists, statisticians, clinical trialists, epidemiologists, psychologists, clinical researchers, clinicians, implementation scientists, evidence synthesis experts, representatives from the Enhancing Quality and Transparency of Health Research Network, and journal editors.

Controlled direct effect:

The exposure’s effect on the outcome if a given mediator were fixed at a constant level uniformly across the entire study population.

Causal directed acyclic graph:

A graphic approach for representing causal relationships between variables and a method for identifying confounding variables that should be adjusted when estimating causal effects (see Figure).

Disjunctive cause criterion:

A criterion that recommends adjusting for all covariates that are causes of the exposure, outcome, or both when the underlying causal structure is unknown and only limited knowledge is available.

Mechanism:

The causal process by which an exposure causes an outcome.

Mediation analysis:

An empirical method used to explain how an exposure causes an outcome.

Mediator:

A variable that may be affected by an exposure and may in turn affect an outcome.

Moderator:

A variable that alters the direction or magnitude of the effect of an exposure on an outcome.

Natural direct effect:

The exposure’s effect on the outcome if a given mediator were fixed at its natural value (defined as the value it would take under a given fixed level of the exposure).

Natural indirect effect:

An effect on the outcome that is caused by the exposure’s effect on a given mediator and that mediator’s subsequent effect on the outcome.

Path-specific effect:

An effect that captures how much of the exposure’s effect on a given outcome is mediated through intermediate variables along 1 or multiple pathways.

Spillover effect:

When the outcome of a participant in a study is affected by the intervention status of other participants in the same study.

Total effect:

The entire effect of the exposure on the outcome that encompasses all indirect and direct effects.

Unmeasured confounder:

An unmeasured variable that is associated with the exposure, mediator, or outcome.

Methods

The AGReMA initiative followed the Enhancing Quality and Transparency of Health Research (EQUATOR) methodological framework for the development of reporting guidelines,⁸ which included: (1) review of systematic reviews of reporting practices; (2) conducting a Delphi survey; (3) having a consensus meeting with methodologists, statisticians, clinical trialists, epidemiologists, psychologists, clinical researchers, clinicians, implementation scientists, evidence synthesis experts, representatives from the EQUATOR Network, and journal editors (n = 19; June-November 2019); and (4) conducting an external review and pilot test. This section provides a summary of the methods (a flow diagram of the checklist development process appears in eFigure 1 in the Supplement). Additional details can be found in the protocol.⁹ The University of New South Wales human research ethics advisory panel provided ethical approval (HC16599). All participants provided electronic informed consent prior to commencing the first Delphi round.

Systematic Reviews of Relevant Evidence on Reporting Mediation Analyses

A previously reported overview of 54 systematic reviews of studies that used mediation analyses found that incomplete reporting impeded interpretation, quality appraisal, reproducibility, and meta-analytic synthesis.⁶ These findings were supported by other systematic reviews of mediation analyses of randomized trials⁷ and observational studies,^10,11 and thus demonstrated the need for a reporting guideline. With assistance from a medical librarian, we conducted a separate scoping search of MEDLINE, PsycINFO, and PubMed (each database searched from inception to March 2019) to identify textbooks and reports that provide guidance on the reporting of mediation studies. We also searched the reference lists of included articles for relevant reports. These reviews, textbooks, and reports were used to identify poorly reported items that were summarized and categorized into themes to be considered by the Delphi panel.

International Delphi Survey

Forty international experts in developing methodological frameworks for mediation analyses or in developing application of mediation analyses for clinical research were invited to participate in a Delphi survey. Nineteen experts agreed to participate and contributed to all 3 Delphi rounds (eTables 1-2 in the Supplement). The Delphi panelists were asked (1) to rate the importance of a list of items generated from the previous systematic reviews, textbooks, reports, and existing reporting guidance for inclusion in AGReMA; (2) to contribute additional items when possible; and (3) to provide suggestions for item refinement. The panel reached consensus on 34 reporting items for study design, analytic procedures, and effect estimates; 3 items were rated as “optional”; and 60 qualitative comments were provided for item refinement and prioritization.¹² The detailed methods and results of the Delphi study have been reported.¹³

Consensus Meeting

A face-to-face consensus meeting was organized to consolidate the final list of reporting items. Due to international travel restrictions imposed by the COVID-19 pandemic, the planned face-to-face meeting was replaced with an online meeting held over 2 days (April 28-29, 2020). A purposeful sample of 15 key experts in methodological development, application of mediation analyses, or reporting of guideline development participated in the meeting (eTables 1-2 in the Supplement). All items from the Delphi survey were reviewed alongside newly suggested items from the consensus panel. The decision rules that were used to guide the consensus meeting and a summary of the anonymized meeting notes appear in eAppendix 1 in the Supplement.

Mediation analyses are often secondary analyses (eg, after primary analysis of a randomized clinical trial) and may be reported within the primary article or as stand-alone reports. To reflect this distinction, we created 25-item (long form) and 9-item (short form) checklists. The long form is intended for reports that primarily focus on the results of mediation analyses, and the short form is intended for reports that primary focus on the principal findings of a randomized trial or observational study along with a short section for mediation analyses. The consensus group rated the importance of each AGReMA item for inclusion in the 9-item short form using a 10-point Likert scale (0 = not important; 9 = critically important), and participants were invited to provide comments as free text.

We calculated the median scores for each item and plotted the distribution of the ratings using histograms. We included items that had a median score greater than 7 and excluded items with a median score of 7 or less. Detailed results appear in eAppendix 2 in the Supplement. This process was not prespecified in the protocol because the idea of creating a short-form checklist was introduced during the development process.

Final Consultation (External Review and Pilot Test)

After reaching consensus, draft versions of the long- and short-form checklists were circulated to all members for comments and edits. The checklists were then pilot tested in November 2020 among peers of the internal steering committee and externally reviewed by 21 expert methodologists and potential users of AGReMA for clarification and specific checklist item wording. During the pilot testing, we asked participants to use the checklists and to provide general feedback on accessibility and usability, and to identify possible reporting items that might have been overlooked. We also asked for specific feedback about the utility and understandability of each item. The characteristics of the participants for the external review and pilot testing appear in eTable 1 and eTable 2 in the Supplement. After this process, all AGReMA members approved and agreed on the final AGReMA statement.

Results

Checklist Items and Explanation

The international consensus process produced a 25-item AGReMA checklist statement and a 9-item AGReMA short-form (AGReMA-SF). The AGReMA-SF is a subset of items from the standard checklist that were considered essential for reporting mediation analyses within reports of randomized trials or observational studies. A decision tree to help users select the appropriate checklist version of AGReMA appears in eFigure 2 in the Supplement.

All items of the AGReMA checklist statement appear in Table 1. The following section provides brief explanations for each AGReMA item and, when possible, evidence that supports the inclusion of each item is referenced. When evidence was not available, the inclusion of the item was supported by the expert consensus panel. The items that are included in the AGReMA-SF checklist appear in Table 2 and are marked with an asterisk (objectives, effects of interest, causal assumptions, measurement, statistical methods, participants, outcomes and estimates, limitations, and interpretation). Excerpts of exemplar reporting will be provided on a public website (https://agrema-statement.org) as reporting standards improve.

Table 1.

A Guideline for Reporting Mediation Analyses (AGReMA) Long-Form Checklist^a

Section and topic	Item No.	Item description
Title and abstract
Title	1	Identify that the study uses mediation analyses
Abstract	2	Provide a structured summary of the objectives, methods, results, and conclusions specific to mediation analyses
Introduction
Background and rationale	3	Describe the study background and theoretical rationale for investigating the mechanisms of interest Include supporting evidence or theoretical rationale for why the intervention or exposure might have a causal relationship with the proposed mediators Include supporting evidence or theoretical rationale for why the mediators might have a causal relationship with the outcomes
Objectives	4	State the objectives of the study specific to the mechanisms of interest The objectives should specify whether the study aims to test or estimate the mechanistic effects
Methods
Study registration	5	If applicable, provide references to any protocols or study registrations specific to mediation analyses and highlight any deviations from the planned protocol
Study design and source of data	6	Specify the design of the original study that was used in mediation analyses and where the details can be accessed, supported by a reference If applicable, describe study design features that are relevant to mediation analyses
Participants	7	Describe the target population, eligibility criteria specific to mediation analyses, study locations, and study dates (start of participant enrollment and end of follow-up)
Sample size	8	State whether a sample size calculation was conducted for mediation analyses If so, explain how it was calculated
Effects of interest	9	Specify the effects of interest
Assumed causal model	10	Include a graphic representation of the assumed causal model including the exposure, mediator, outcome, and possible confounders
Causal assumptions	11	Specify assumptions about the causal model
Measurement	12	Clearly describe the interventions or exposures, mediators, outcomes, confounders, and moderators that were used in the analyses Specify how and when they were measured, the measurement properties, and whether blinded assessment was used
Measurement levels	13	If relevant, describe the levels at which the exposure, mediator, and outcome were measured
Statistical methods	14	Describe the statistical methods used to estimate the causal relationships of interest This description should specify analytic strategies used to reduce confounding, model building procedures, justification for the inclusion or exclusion of possible interaction terms, modeling assumptions, and methods used to handle missing data Provide a reference to the statistical software and package used
Sensitivity analyses	15	Describe any sensitivity analyses that were used to explore causal or statistical assumptions and the influence of missing data
Ethical approval	16	Name the institutional research board or ethics committee that approved the study Provide a description of participant informed consent or ethics committee waiver of informed consent
Results
Participants	17	Describe baseline characteristics of participants included in mediation analyses Report the total sample size and number of participants lost during follow-up or with missing data
Outcomes and estimates	18	Report point estimates and uncertainty estimates for the exposure-mediator and mediator-outcome relationships If inference concerning the causal relationship of interest is considered feasible given the causal assumptions, report the point estimate and uncertainty estimate
Sensitivity parameters	19	Report the results from any sensitivity analyses used to assess robustness of the causal or statistical assumptions and the influence of missing data
Discussion
Limitations	20	Discuss the limitations of the study including potential sources of bias
Interpretation	21	Interpret the estimated effects considering the study’s magnitude and uncertainty, plausibility of the causal assumptions, limitations, generalizability of the findings, and results from relevant studies
Implications	22	Discuss the implications of the overall results for clinical practice, policy, and science
Other information
Funding and role of sponsor	23	List all sources of funding or sponsorship for mediation analyses and the role of the funders/sponsors in the conduct of the study, writing of the manuscript, and decision to submit the manuscript for publication
Conflicts of interest and financial disclosures	24	State any conflicts of interest and financial disclosures for all authors
Data and code	25	Authors are encouraged to provide a statement for sharing data and code for mediation analyses

^aDesigned for articles that report primary mediation analyses of randomized trials or observational studies or those that report secondary mediation analyses as the primary focus of an article. Republished with permission from the AGReMA group. This checklist is copyrighted by the AGReMA group under the Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported (CC BY-NC-ND 3.0) license.

Table 2.

A Guideline for Reporting Mediation Analyses Short-Form (AGReMA-SF) Checklist^a

Section and topic	Item No.	Item description
Introduction
Objectives	1	State the objectives of the study, specific to the mechanisms of interest The objectives should specify whether the study aims to test or estimate the mechanistic effects
Methods
Effects of interest	2	Specify the effects of interest
Causal assumptions	3	Specify assumptions about the causal model
Measurement	4	Clearly describe the interventions or exposures, mediators, outcomes, confounders, and moderators that were used in the analyses Specify how and when they were measured, the measurement properties, and whether blinded assessment was used
Statistical methods	5	Describe the statistical methods used to estimate the causal relationships of interest This description should specify analytic strategies used to reduce confounding, model building procedures, justification for the inclusion or exclusion of possible interaction terms, modeling assumptions, and methods used to handle missing data Provide reference to the statistical software and package used
Results
Participants	6	Describe baseline characteristics of participants included in mediation analyses Report the total sample size and number of participants lost during follow-up or with missing data
Outcomes and estimates	7	Report point estimates and uncertainty estimates for the exposure-mediator and mediator-outcome relationships If inference concerning the causal relationship of interest is considered feasible given the causal assumptions, report the point estimate and uncertainty estimate
Discussion
Limitations	8	Discuss the limitations of the study including potential sources of bias
Interpretation	9	Interpret the estimated effects considering the study’s magnitude and uncertainty, plausibility of the causal assumptions, limitations, generalizability of the findings, and results from relevant studies

^aDesigned for articles that report secondary mediation analyses within a primary report of a randomized trial or observational study and may be used alongside a main reporting guideline such as the Consolidated Standards of Reporting Trials or the Strengthening the Reporting of Observational Studies in Epidemiology. Republished with permission from the AGReMA group. This checklist is copyrighted by the AGReMA group under the Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported (CC BY-NC-ND 3.0) license.

Title and Abstract

Item 1. Title

Identify that the study uses mediation analyses.

Explanation ∣

Readers should be able to identify from the title that the study used mediation analyses. Including terms such as mediation analysis (Medical Subject Headings term), mediation, or mediator in the title or as keywords can ensure that mediation studies will be appropriately indexed and identified in literature searches.

Item 2. Abstract

Provide a structured summary of the objectives, methods, results, and conclusions specific to mediation analyses.

Explanation ∣

It is recommended that authors describe (at minimum) the study objectives (ideally supported by a brief statement of background and rationale for the mechanisms of interest), methods (ideally including the setting, participants, sample size, exposure, mediator, outcome, and analytic approach for mediation analyses), results (including point estimates and uncertainty estimates), and the main conclusion.

Introduction

Item 3. Background and Rationale

Describe the study background and theoretical rationale for investigating the mechanisms of interest. Include supporting evidence or the theoretical rationale for why the intervention or exposure might affect the proposed mediators and why the mediators might affect the outcomes.

Explanation ∣

A concise description of the study background should be included to provide context for the subject matter and clinical setting of the study. Most often, mediation analyses will be used to understand the mechanisms by which an intervention or exposure might affect an outcome. It is recommended that authors make clear why mediation analyses helps to answer the substantive scientific question. Describing the theory that underpins the proposed mechanisms of interest, stating why the exposure or intervention is expected to affect the proposed mediator (action theory), and why the mediator is expected to affect the outcome (conceptual theory) is recommended.¹² This type of rationale should reflect each objective and, when possible, should be supported with empirical or qualitative evidence.

Item 4. Objectives*

State the objectives of the study specific to the mechanisms of interest. The objectives should specify whether the study aims to test or estimate the mechanistic effects.

Explanation ∣

The background section should end with a clear statement of the main objectives of mediation analyses. The objectives should specify whether the aim is (1) to test the presence of an indirect or direct effect or (2) to estimate the magnitude of an indirect or direct effect. The objectives can also help to declare whether the aim of mediation analyses is explanatory (to explain what mediates a causal relationship) or interventional (to ask questions about possible causal mechanisms of hypothetical interventions that target the exposure or mediator).⁵ When mediation analyses are used to answer a secondary question, authors should clearly state the objectives but note that the objective of mediation analyses is secondary and place it within the context of the primary objective.

Methods

Item 5. Study Registration

If applicable, provide references to any protocols or study registrations specific to mediation analyses and highlight any deviations from the planned protocol.

Explanation ∣

If the protocol for the mediation analyses is registered (either within an overall analysis plan or as a separate secondary analysis plan), authors should report the name of the register, repository, or journal where the protocol was registered and provide the registration number or digital object identifier. If the study is not registered or linked to a published protocol, authors should explicitly declare the exploratory nature of the mediation analyses.

Item 6. Study Design and Source of Data

Specify the design of the original study that was used in the mediation analyses and where the details can be accessed, supported by a reference. If applicable, describe study design features that are relevant to mediation analyses.

Explanation ∣

Mediation analyses are often applied to data from randomized trials and observational (cohort and case-control) studies.¹ It is important for the mediation study to provide sufficient detail on design features, preferably with reference to a publication that contains detail about the original study that generated the data. In rare instances in which the original randomized trial or observational study cannot be referenced, the report for mediation analyses should provide greater detail on the study design and data sources.

Different study designs require different sets of assumptions for the estimation of indirect and direct effects in mediation analyses (see item 11). For example, in a randomized trial, it would be considered appropriate to assume that the intervention-mediator effects and the intervention-outcome effects are not confounded because of random allocation of the intervention. This is generally not the case in observational designs. Design variations within observational studies, such as case-control and cohort designs, can require different analytic approaches that each require different assumptions.¹⁴ Therefore, it is important to provide a clear description of the original study design and data sources so the potential risks of bias can be assessed.

Item 7. Participants

Describe the target population, eligibility criteria specific to mediation analyses, study locations, and study dates (start of participant enrollment and end of follow-up).

Explanation ∣

Like most inferential studies, mediation analyses will study a sample of a defined target population. To provide an indication of representativeness, authors are recommended to provide a clear definition of the target population, factors that determine eligibility and recruitment into the study sample, and where (eg, geographic location and setting) and when (eg, range of dates) the study took place. Doing so will allow readers to gauge whether the findings from the mediation analyses are generalizable to the target population of interest and assist systematic reviewers in assessing study heterogeneity.

Item 8. Sample Size

State whether a sample size calculation was conducted for the mediation analyses. If so, explain how it was calculated.

Explanation ∣

Sample size calculations for mediation analyses are not commonly conducted or reported,^6,7 partly because sample size calculations are complex and dependent on study design and analytic methods.¹⁵ If a sample size calculation was conducted, authors should report the calculation method and the estimates used in the calculation (eg, the effect of the exposure on the mediator and residual mediator variance, the effects of the exposure and the mediator on the outcome and residual outcome variance, significance level, and desired power) along with any assumptions. If possible, providing a reference to the software that was used can facilitate reproducibility.

Item 9. Effects of Interest*

Specify the effects of interest.

Explanation ∣

Depending on the research question and the study objectives, investigators will aim to test or estimate 1 or more of the following possible effects: exposure-mediator effect, mediator-outcome effect, controlled direct effect, natural direct and indirect effects,¹⁶ interventional direct and indirect effects,¹⁷ or path-specific effects.¹ For example, Boers et al¹⁸ reported a clinical definition of a natural indirect effect as the possible causal relationship between endovascular therapy and functional outcome that is explained by a treatment-related reduction in follow-up infarct volume.

As a more detailed definition, Stensrud and Strohmaier¹⁹ reported their natural indirect effect as a comparison of the risk of a cardiovascular event when blood pressure values were those that would occur with intensive therapy vs the risk of a cardiovascular event when blood pressure values were those that would occur with standard therapy but in fact occurred during receipt of intensive therapy.

Because the chosen effect of interest will require a specific set of assumptions, drive the analytic method, and guide interpretation, it is essential for authors to clearly report the hypothesized effect that is most relevant to the study objectives (item 4).⁵ In some instances, investigators will have multiple study objectives and multiple effects of interest. If so, it is recommended that authors link the study objectives to the possible effects of interest.

Item 10. Assumed Causal Model

Include a graphic representation of the assumed causal model including the exposure, mediator, outcome, and possible confounders.

Explanation ∣

For most mediation analyses, investigators will apply field-specific knowledge, theories, and assumptions to propose an assumed causal model. The assumed causal model should be transparently described because it can influence how mediation analyses are conducted, and thereby influence the results and their interpretation. One practical and effective method of communicating the assumed causal model is the use of causal directed acyclic graphs (Figure).²⁰

Figure. Causal Directed Acyclic Graph Depicting Typical Variables and Relationships That Are Relevant to Mediation Analyses.

A confounder of the association between an exposure and a mediator or between an exposure and an outcome is a preexposure variable that is associated with the exposure and with the mediator or outcome, respectively. A confounder of the association between a mediator and an outcome is a premediator variable (possibly affected by the exposure) that is associated with the mediator and outcome. Because confounders can distort associations, controlling for confounders of the exposure-mediator, exposure-outcome, and mediator-outcome associations is important in mediation analyses. A collider on a path in the causal directed acyclic graph between 2 variables is a variable that is affected by both variables. Standard adjustment for a collider typically introduces selection bias and special care may be needed when controlling for colliders. Effect modification (interaction) cannot be depicted in a standard directed acyclic graph.

Causal directed acyclic graphs for mediation analyses should include nodes that represent the intervention or exposure, the mediator, the outcome, possible confounders of the relationships between these variables, and unidirectional arrows that depict the assumed causal relationships between the displayed variables. It is often useful to include both measured and unmeasured variables when there may be confounding by both types and to specify which variables were adjusted for in the analysis. It is also important to indicate possible collider variables are represented in the assumed causal model because conditioning on a collider can induce selection bias.

Item 11. Causal Assumptions*

Specify assumptions about the causal model.

Explanation ∣

It is important to be explicit about the assumptions of a causal model because they guide the analytic approach, expose possible sources of bias, and help determine the extent to which an estimate can be interpreted as a possible causal relationship. For example, stating which unmeasured confounders of the exposure-mediator, exposure-outcome, and mediator-outcome relationships are considered important and could guide the sensitivity analyses (see item 15) and allow the reader to gauge how unmeasured confounders would influence the interpretation of the estimates.

Clearly outlining the temporal precedence of the variables in a mediation model is also important for assessing the direction of hypothesized causal relationships and the possibility of reverse causation. Critical assumptions in mediation analyses, such as no unmeasured confounding, can be expressed in the form of causal directed acyclic graphs (item 10),²¹ whereas assumptions such as effect modification, positivity, and consistency will be better expressed as written statements.²²

Item 12. Measurement*

Clearly describe the interventions or exposures, mediators, outcomes, confounders, and moderators that were used in the analyses. Specify how and when they were measured, the measurement properties, and whether blinded assessment was used.

Explanation ∣

All variables included in mediation analyses, such as the interventions or exposures, mediators, outcomes, and confounders, should be clearly identified and unambiguously defined. Authors should state how each variable was measured and describe the measurement tool (eg, a survey instrument such as the 36-Item Short Form Health Survey) that was used. Authors should clearly specify the beginning of follow-up (time zero) relative to when individuals met the eligibility criteria and when the intervention or exposure was initiated,²³ and report the relative timing of the exposure, mediator, and outcome measurements so that the possibility of immortal time bias and temporal precedence can be assessed.

The goal should be to provide sufficient detail so that others can replicate the study using the same variables and systematic reviewers can include or exclude studies or group studies based on the measured variables. When the exposure is an intervention, the Template for Intervention Description and Replication checklist²⁴ should be used with the AGReMA checklist. Because measurement error can introduce bias in mediation analyses,²⁵ it is important to report relevant measurement properties of the assessment or measure that was used (eg, reliability). In addition, authors should describe the extent to which participants and study personnel were masked to the intervention allocation or exposure level. This detail will allow for the assessment of observer and detection bias.²⁶

Item 13. Measurement Levels

If relevant, describe the levels at which the exposure, mediator, and outcome were measured.

Explanation ∣

In some situations, mediation analyses will be applied to settings in which individuals are clustered within groups such as households, schools, hospitals, and countries. For example, in a cluster-randomized trial, researchers may study the effect of a hospital-level intervention on mediators and outcomes measured at the individual level. The data are considered multilevel or clustered because the data from individuals within 1 hospital may be more similar to each other than those from other hospitals and thus correlated. In these settings, authors should describe whether the exposures, mediators, and outcomes were assigned or measured at the group or individual level. Authors are also encouraged to describe how clustering was accounted for with regard to within- and between-cluster heterogeneity,²⁷ and possible spillover effects if relevant,²⁸ for the estimation of direct and indirect effects.

Item 14. Statistical Methods*

Describe the statistical methods used to estimate the causal relationships of interest. This description should specify the analytic strategies used to reduce confounding, model building procedures, justification for the inclusion or exclusion of possible interaction terms, modeling assumptions, and the methods used to handle missing data. Provide a reference to the statistical software and package used.

Explanation ∣

Broadly there are 2 major traditions for conducting mediation analyses: those deriving from the causal steps of Baron and Kenny or with a product and difference-of-coefficients framework²⁹ and those from the counterfactual-based framework.^1,30 Authors might indicate which 1 of these 2 frameworks were used in their mediation analyses. They also should clearly specify which specific methods within the chosen framework were used (eg, by providing a reference). Reporting the name and version of the statistical software and any specific packages can be useful for reproducing analyses.

Most mediation analyses will use a theory-driven approach to identify and adjust for a sufficient set of confounders of the exposure-mediator, exposure-outcome, and mediator-outcome associations. Authors should report how confounders were identified, for example, through the use of causal directed acyclic graphs,²¹ the disjunctive cause criterion,³¹ or when data-driven, use of variable selection procedures such as stepwise testing strategies or penalization methods in models for the mediator and outcome. It is also useful to report confounders that were identified in the assumed causal model but were not measured or adjusted for (see items 10 and 11).

Most mediation analyses will use regression models for the mediator and the outcome. Depending on the nature of these variables, investigators will select the most appropriate regression model, such as Cox regression for time-to-event mediators and outcomes or logistic regression for binary mediators and outcomes. Authors should clearly report the functional form and specification of the regression models that were used to model the mediators and outcomes and report any modeling assumptions that were made. If a variable selection procedure was used or if interactions were modeled to improve model flexibility, authors should report these so that readers can assess the appropriateness of the models that eventually inform the estimation of the direct and indirect effects.

Similar to most applied research, missing data are common in mediation analyses, and the way in which missing data are handled can affect the estimates of the direct and indirect effects. Depending on the amount of missing data and missingness patterns, various imputation methods may be used. It is important that authors state whether the data were imputed and, if so, report detailed information about the selected method for handling missing data.³²

Item 15. Sensitivity Analyses

Describe any sensitivity analyses that were used to explore causal assumptions, statistical assumptions, or both, and the influence of missing data.

Explanation ∣

Broadly, there are 2 types of assumptions in mediation analyses: causal and statistical. The causal assumptions refer to the underlying theoretical model being investigated (items 10 and 11). For example, investigators might assume that there is no residual confounding of the exposure-mediator, exposure-outcome, and mediator-outcome relationships. It is also common to make assumptions about the direction of causal relationships between mediators or the absence of common causes of multiple mediators.³³ If sensitivity analyses (such as the mediational E-value³⁴) are used to explore violation of such assumptions, authors should describe and cite the approach that was used.

Although most causal assumptions cannot be empirically verified, statistical assumptions that are inherent to modeling procedures can be empirically verified. For example, determining how well a selected model fits the observed data is often assessed using residual plots. To enable readers to understand how model fit was assessed, authors should report which goodness-of-fit assessment was used to assess the working models. Because the results from mediation analyses may vary depending on the imputation method used to account for missing data, any sensitivity analyses used to assess the method of handling missing data should be reported.

Item 16. Ethical Approval

Name the institutional research board or ethics committee that approved the study and provide a description of participant informed consent or an ethics committee waiver of informed consent.

Explanation ∣

It is expected that most studies that use mediation analyses will have sought ethical approval from an institutional research board or ethics committee. This may be approval for the original randomized trial or observational study, or a separate approval for the mediation analyses. The details of the approval and how informed consent was obtained or waived should be clearly reported.

Results

Item 17. Participants*

Describe the baseline characteristics of the participants included in the mediation analyses and report the total sample size and the number of participants lost during follow-up or with missing data.

Explanation ∣

To allow readers of mediation analyses to understand the characteristics of the sample and to gauge the generalizability of the findings, the baseline characteristics of the sample (demographics, clinical features, mediator, and outcome) should be reported. It is also important to report the total sample size and the number of participants lost during follow-up along with the amount and pattern of missing data for the mediators, outcomes, and possible confounders because losses to follow-up and missing data can introduce bias (see item 14). Reporting how the baseline characteristics of those lost to follow-up or with missing data compared with the participants analyzed can provide readers with a sense of how likely it is for selection bias to influence the results.

When mediation analyses are embedded in randomized trials or observational studies, it may not be sufficient to describe only the overall participants included in the primary study because the variables required for the mediation analyses may have been collected only in a subsample of the primary study sample (by intention or because of missing data). In these circumstances, it is important to report the subsample that is included in mediation analyses. It may also be helpful to report the total effect (exposure-outcome association without considering the mediator) obtained from the primary study sample compared with the total effect from the subsample used in the mediation analyses. When mediation analyses are reported as secondary analyses within a main report of a randomized trial or observational study, or when word count is limited in the main text, it may be sufficient to report this item within a supplement.

Item 18. Outcomes and Estimates*

Report point estimates and uncertainty estimates for the exposure-mediator and mediator-outcome relationships. If inference concerning the causal relationship of interest is considered feasible given the causal assumptions, report the point estimate and uncertainty estimate.

Explanation ∣

Selecting which causal relationships to report from mediation analyses will depend on the study objectives (item 4). In most cases, the natural direct and indirect effects are recommended when the aim is to explain the causal relationship between an exposure and an outcome through 1 or more mediators (eg, the natural indirect effect of intensive blood pressure therapy on cardiovascular events mediated through low diastolic blood pressure had a hazard ratio of 1.12 [95% CI, 1.06-1.18] and the natural direct effect not mediated through low diastolic blood pressure had a hazard ratio of 0.63 [95% CI, 0.50-0.78]).¹⁹ If the study objective is to estimate the causal relationship between an exposure and an outcome while a mediator is fixed at a constant level uniformly across the population, the controlled direct effect is recommended (eg, the causal relationship between ablation surgery and returning to sinus rhythm if no patient in the target population had the left atrial appendage removed had a hazard ratio of 0.14 [95% CI, 0.02-0.25] on the probability difference scale).³⁵

The estimation of exposure-mediator and mediator-outcome relationships will often require weaker assumptions than the estimation of direct and indirect effects. For this reason, as well as to provide more insight into the possible mechanisms of interest, authors should always report relevant estimates for the exposure-mediator and mediator-outcome relationships. When the necessary causal assumptions are thought to be plausible, authors should report unstandardized estimates, standardized estimates, or both, of direct and indirect effects along with their standard errors or 95% CIs.³⁶ The scale on which these effects are measured (eg, mean difference, risk difference, risk ratio, odds ratio, hazard ratio) must also be clearly reported. Authors may choose to report the proportion mediated (or eliminated) along with their 95% CIs as a descriptive summary of the results. Because there can be considerable uncertainty around the proportion mediated, especially in small samples, keeping the focus on the indirect and direct effects of interest is recommended.

Item 19. Sensitivity Parameters

Report the results from any sensitivity analyses used to assess the robustness of causal assumptions, statistical assumptions, or both, and the influence of missing data.

Explanation ∣

The validity of most mediation analyses will depend on unverifiable causal assumptions. The main assumption is no unmeasured confounding. Reporting the results of any analyses that explore the sensitivity of the results regarding violation of the no unmeasured confounding assumption can allow the reader to judge the robustness of the findings. Several metrics can be reported, such as the mediational E-value,³⁴ or sensitivity parameters that quantify how much residual confounding there would need to be to invalidate the estimated direct and indirect effect.³⁰ Authors should be clear about the metric used and provide a brief interpretation in the context of the main findings.

If other sensitivity analyses are used to explore assumptions about the study design, measurement tools, statistical models, and missing data, the results of these analyses should be reported in the supplementary material. This will help readers gauge the plausibility of the assumptions and the robustness of the findings.

Discussion

Item 20. Limitations*

Discuss the limitations of the study, including potential sources of bias.

Explanation ∣

Studies that use mediation analyses may have a number of limitations such as failure to account for unmeasured confounding,³⁷ measurement error,²⁵ model misspecification,³⁸ selection bias,³⁹ and missing data.⁴⁰ Authors should state any limitations and comment on how they might affect the validity and veracity of the main findings.

If a sensitivity analysis was used to explore the effect of a limitation (items 15 and 19), the results should be discussed considering the main findings. Limitations should be clearly stated, and when relevant, discussed in the context of other studies. When mediation analyses are reported as secondary analyses within a main report, or when the word count is limited in the main text, it may be sufficient to report the limitations in a supplement.

Item 21. Interpretation*

Interpret the estimated effects considering their magnitude and uncertainty, plausibility of the causal assumptions, limitations, generalizability of the findings, and results from relevant studies.

Explanation ∣

The main findings with respect to the main objectives should be summarized in a concise paragraph. An important aspect of interpreting estimates from mediation analyses is appraising whether the estimate can have a possible causal interpretation. This will depend on how reasonable the causal assumptions are (item 11), possibly supplemented with results from sensitivity analyses (item 19) and other limitations (item 20).

Authors should provide a balanced discussion of these issues to allow the reader to judge whether the estimates can be given a causal interpretation. The interpretation should also be set in the context of any previously identified theoretical or evidence-based rationale for mediation analyses, particularly when the findings support or challenge theory. The generalizability of the overall findings should also be discussed to guide the application of the findings into clinical practice, if appropriate. When the mediation analyses are part of the secondary study objective, the interpretation might focus on the direct and indirect effects of interest in the context of the primary findings.

Item 22. Implications

Discuss the implications of the overall results for clinical practice, policy, and science.

Explanation ∣

Authors should consider discussing whether the findings may influence clinical practice, policy, or future research while considering the limits of mediation analyses. These implications may, for example, suggest how an intervention or policy could be delivered to specifically target (or avoid targeting) particular mediators. Implications for research might suggest how interventions could be refined to improve efficiency or efficacy in future studies.

Other Information

Item 23. Funding and Role of Sponsor

List all sources of funding or sponsorship for mediation analyses and the role of the funders/sponsors in the conduct of the study, writing of the manuscript, and decision to submit the manuscript for publication.

Explanation ∣

Information about study funding and support is important for helping readers identify potential conflicts of interest or possible influence. Authors should identify and declare all sources of study funding and support. Authors should report the name of the persons or entities supported, the name of the funder, and the grant or award number if available. Authors should explicitly outline the roles and responsibilities of the funder/sponsor in the study design, conduct, data analysis and interpretation, manuscript writing, and dissemination of results and should describe whether the funder/sponsor had input into the final decision regarding any of these aspects. If the funder/sponsor was not involved or had no influence, authors should specifically report this.

Item 24. Conflicts of Interest and Financial Disclosures

State any conflicts of interest and financial disclosures for all authors.

Explanation ∣

Conflicts of interests can be a source of bias.⁴¹ These conflicts include financial relationships (such as employment, consultancies, stock ownership or options, honoraria, patents, and paid expert testimony), personal relationships or rivalries, academic competition, and intellectual beliefs. Financial conflicts of interest are associated with publication of research outcomes that favor the financial interest. Although the presence of a relationship or activity does not always indicate a problematic influence, conflicts should be transparently declared to allow readers to make their own judgments.

All authors should disclose any relationships or activities that might bias the study conduct and reporting. The International Committee of Medical Journal Editors has developed a disclosure form to facilitate and standardize authors’ disclosures.

Item 25. Data and Code

Authors are encouraged to provide a statement for sharing data and code for mediation analyses.

Explanation ∣

Availability of data and code is essential for reproducing and replicating study findings. Open access to data and code facilitates validation of analytic methods during and after peer review. Furthermore, with the availability of various analytic options for mediation analyses, sharing data and code makes modeling procedures, assumptions, and estimation procedures transparent to the reviewer and reader.

If possible, data should be shared in an accessible, secure, and reliable database. Shared data should adhere to the Findable, Accessible, Interoperable and Reusable guiding principles⁴² and, when possible, have a corresponding digital object identifier. At a minimum, a data and code availability statement should be provided within the report.

Discussion

The AGReMA statement provides international consensus-based guidance on items that should be reported in studies that use mediation analyses. The scope of the AGReMA statement covers primary and secondary mediation analyses of randomized trials and observational studies, and it is intended to be general so that it can guide the reporting of most mediation analyses. Earlier approaches to mediation analyses, including the causal steps of Baron and Kenny or the product and difference-of-coefficients framework,²⁹ are valid under restricted conditions (linear models without interactions). In contrast, causal mediation analyses based on the counterfactual-based framework can be valid under general conditions (arbitrary linear and nonlinear models) and explicitly outline the causal assumptions that are required for making causal inferences.^1,30 Although terms such as direct effect, indirect effect, and path-specific effects are conventional terminology for mediation analyses, they should be interpreted with caution in both observational designs and randomized trials because causal assumptions may be unmet and it may not be possible to establish causal inferences.

The AGReMA project was designed to provide a minimum set of recommendations for reporting. Therefore, authors are encouraged to report additional details that are relevant to their study and readership when possible. The AGReMA-SF checklist is composed of essential items and was developed to guide the reporting of secondary mediation analyses that are reported within randomized trial or observational study reports. However, when possible (and especially when the total effect has been reported in a separate article), it is better to use the long-form checklist.

The purpose of the AGReMA statement is to improve completeness, consistency, and accuracy in reporting. It is not designed to guide conductor to be used as a risk of bias tool. However, it could enable systematic reviewers to assess risk of bias by improving the reporting of relevant information. The AGReMA working group will aim to maximize the awareness and uptake of AGReMA by liaising with relevant journal editors and funding agencies to encourage the endorsement of the AGReMA checklists. To improve accessibility, the AGReMA checklists will be made available on an open web domain (https://agrema-statement.org) and indexed in the EQUATOR Network website.

Limitations

This guideline and the guideline development process have several limitations. First, participants of the Delphi process and consensus meetings were purposefully selected based on expertise and familiarity with mediation analyses and scientific reporting. Although this select group of participants may not represent potential users of AGReMA, the consolidated checklist was externally reviewed and pilot tested by a broad group of 21 experts and potential users (eTables 1-2 in the Supplement), and their feedback was used to adjust the guideline.

Second, approaches to mediation analyses are grounded in 2 distinct traditions. Proponents of both analytic traditions were included as participants and AGReMA aims to provide guidance for both approaches. Even though the intention was to include equal representation of participants from both analytic traditions, the emphasis in the reporting guidance may have been influenced by the composition of the panel.

Third, because of travel and social contact restrictions from the COVID-19 pandemic, the consensus meeting was conducted online rather than face-to-face. This format may have inhibited a more detailed and fluid discussion, but attempts were made to mitigate these issues by structuring the meeting so that participants were encouraged to discuss, introduce, and remove items. Smaller group discussions also took place after the 2-day meeting.

Conclusions

The AGReMA statement provides recommendations for reporting primary and secondary mediation analyses of randomized trials and observational studies. Improved reporting of studies that use mediation analyses could facilitate peer review and help produce publications that are complete, accurate, transparent, and reproducible.

Key Points.

Question

What information should be reported in studies that include mediation analyses of randomized trials and observational studies?

Findings

An international Delphi and consensus process (using the Enhancing Quality and Transparency of Health Research methodological framework) generated a 25-item reporting guideline for primary reports of mediation analyses and a 9-item short form for secondary reports of mediation analyses.

Meaning

Using the 25-item or 9-item reporting guideline may facilitate peer review and could help ensure that studies using mediation analyses are completely, accurately, and transparently reported.

Appendix

The AGReMA group authors:

A. Russell Localio, PhD; Ludo van Amelsvoort, PhD; Eliseo Guallar, PhD; Judith Rijnhart, PhD; Kimberley Goldsmith, PhD; Amanda J. Fairchild, PhD; Cara C. Lewis, PhD; Steven J. Kamper, PhD; Christopher M. Williams, PhD; Nicholas Henschke, PhD.

Affiliations of the AGReMA group authors:

School of Medicine and Public Health, University of Newcastle, Callaghan, Australia (Williams); Department of Biostatistics, Epidemiology, and Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia (Localio); Associate Editor, Annals of internal Medicine (Localio); Faculty of Health, Medicine, and Life Sciences, Maastricht University, Maastricht, the Netherlands (van Amelsvoort); Assoicate Editor, Journal of Clinical Epidemiology (van Amelsvoort); Welch Center for Prevention, Epidemiology, and Clinical Research, Bloomberg School of Public Health, Johns Hopkins University, Baltimore, Maryland (Guallar); Deputy Editor, Annals of internal Medicine (Guallar); Department of Epidemiology and Data Science, Amsterdam Public Health Research Institute, Amsterdam University Medical Center, Amsterdam, the Netherlands (Rijnhart); Institute of Psychiatry, Psychology, and Neuroscience, King’s College London, London, England (Goldsmith); Department of Psychology, University of South Carolina, Columbia (Fairchild); Kaiser Permanente Washington Health Research Institute, Seattle (Lewis); School of Health Sciences, University of Sydney, Sydney, Australia (Kamper); Nepean Blue Mountains Local Health District, Kingswood, Australia (Kamper); School of Public Health, University of Sydney, Sydney, Australia (Henschke).