Skip to main content
. Author manuscript; available in PMC: 2021 Sep 7.
Published in final edited form as: Environ Int. 2020 May 17;141:105736. doi: 10.1016/j.envint.2020.105736

Table 1.

Study evaluation domains with core and prompting questions.

Key concept Domain - Core Question Prompting questions
Reporting Quality 1. Reporting quality - Does the study report information for evaluating the design and conduct of the study for the outcome(s) of interest? Does the study report the following?
  • Critical information necessary to perform study evaluation:
    • Species; test article name; levels and duration of exposure; route (e.g., oral; inhalation); qualitative or quantitative results for at least one outcome of interest
  • Important information for evaluating the study methods:
    • Test animal: strain, sex, source, and general husbandry procedures
    • Exposure methods: source, purity, method of administration
    • Experimental design: frequency of exposure, animal age and lifestage during exposure and at outcome evaluation
    • Outcome evaluation methods: assays or procedures used to measure the outcomes of interest
Risk of Bias 2. Allocation - Were animals assigned to experimental groups using a method that minimizes selection bias? For each study:

  • Did each animal or litter have an equal chance of being assigned to any experimental group (i.e., random allocation)?

  • Is the allocation method described?

  • Aside from randomization, were any steps taken to balance variables across experimental groups during allocation?
3. Observational bias/blinding - Did the study implement measures to reduce observational bias? For each outcome or grouping of outcomes in a study:

  • Does the study report blinding or other methods/procedures for reducing observational bias?

  • If not, did the study use a design or approach for which such procedures can be inferred?

  • What is the expected impact of failure to implement (or report implementation) of these methods/procedures on results?
4. Confounding - Are variables with the potential to confound or modify results controlled for and consistent across all experimental groups? For each study:

  • Are there differences across the treatment groups (e.g., co-exposures, vehicle, diet, palatability, husbandry, health status, etc.) that could bias the results?

  • If differences are identified, to what extent are they expected to impact the results?
5. Selective reporting and attrition - Did the study report results for all prespecified outcomes and tested animals? For each study: Selective reporting bias:

  • Are results presented for all outcomes described in the methods?

Attrition bias:

  • Are all animals accounted for in the results?

  • If there are discrepancies, do authors provide an explanation (e.g., death or unscheduled sacrifice during the study)?

  • If results omissions and/or attrition are identified, what is the expected impact on the interpretation of the results?
Sensitivity 6. Chemical administration and characterization - Did the study adequately characterize exposure to the chemical of interest and the exposure administration methods? For each study:

  • Does the study report the source and purity and/or composition (e.g., identity and percent distribution of different isomers) of the chemical? If not, can the purity and/or composition be obtained from the supplier (e.g., as reported on the website)

  • Was independent analytical verification of the test article purity and composition performed?

  • Did the authors take steps to ensure the reported exposure levels were accurate?
    • For inhalation studies: were target concentrations confirmed using reliable analytical measurements in chamber air?
    • For oral studies: if necessary, based on consideration of chemical-specific knowledge (e.g., instability in solution; volatility) and/or exposure design (e.g., the frequency and duration of exposure), were chemical concentrations in the dosing solutions or diet analytically confirmed?

  • Are there concerns about the methods used to administer the chemical (e.g., inhalation chamber type, gavage volume, etc.)?
7. Exposure timing, frequency, and duration - Was the timing, frequency, and duration of exposure sensitive for the outcome(s) of
interest?		 For each outcome or grouping of outcomes in a study:

  • Does the exposure period include the critical window of sensitivity (if known)?

  • Was the duration and frequency of exposure sensitive for detecting the outcome of interest?
8. Sensitivity and specificity - Are the procedures sensitive and specific for evaluating the outcome(s) of interest? For each outcome or grouping of outcomes in a study:

  • Are there concerns regarding the specificity and validity of the protocols?

  • Are there serious concerns regarding the sample size?

  • Are there concerns regarding the timing of the outcome assessment?
9. Results presentation - Are the results presented in a way that makes the data usable and transparent? For each outcome or grouping of outcomes in a study:

  • Does the level of detail allow for an informed interpretation of the results?

  • Are the data compared or presented in a way that is inappropriate or misleading?
Overall Confidence 10. Overall Confidence - Considering the identified strengths and limitations, what is the overall confidence rating for the outcome(s) of interest? For each outcome or grouping of outcomes in a study:

  • Were concerns (i.e., limitations or uncertainties) related to the reporting quality, risk of bias, or sensitivity identified?

  • If yes, what is their expected impact on the overall reliability and validity of the study results, including (when possible) interpretations of impacts on the magnitude or direction of the reported effects?