Inconsistencies in variable reporting and methods in larval zebrafish behavioral assays

Abstract

New approaches in developmental neurotoxicity (DNT) screening are needed due to the tens of thousands of chemicals requiring hazard assessments. Zebrafish (Danio rerio) are an alternative vertebrate model for DNT testing, but without a standardized protocol for larval behavioral assays, comparison of results among laboratories is challenging. To evaluate the congruence of protocols across laboratories, we conducted a literature review of DNT studies focusing on larval zebrafish behavior assays and cataloged experimental design consistencies. Our review focused on 51 unique method variables in publications where chemical exposure occurred in early development and subsequent larval locomotor evaluation focused on assays that included a light/dark photoperiod transition. We initially identified 94 publications, but only 31 exclusively met our inclusion criteria which focused on parameters that are important to an assay employed by our laboratory. No publication reported 100% of the targeted variables; only 51 to 86% of those variables were reported in the reviewed publications, with some aspects of the experimental design consistent among laboratories. However, no protocol was exactly the same for any two publications. Many of these variables had more than one parameter/design reported, highlighting the inconsistencies among methods. Overall, there is not only a strong need for the development of a standardized testing protocol for larval zebrafish locomotor assays, but there is also a need for a standardized protocol for reporting experimental variables in the literature. Here we include an extensive guideline checklist for conducting larval zebrafish developmental behavior assays.

1. Introduction

Larval zebrafish (Danio rerio) behavior is a widely used endpoint in pharmacology and toxicology (Fontana et al., 2018; Kalueff et al., 2014; Nishimura et al., 2015). The size, fecundity, and affordability of zebrafish make them an ideal test subject for high-throughput assays. In addition, the close relationship between behavior and nervous system function allows researchers to use changes in behavior to detect and interpret perturbations in the development and function of the nervous system (de Esch et al., 2012a). However, one challenge faced by researchers using zebrafish behavioral approaches is the variability of results within and across laboratories (e.g., (Ball et al., 2014; Gustafson et al., 2012; Hsieh et al., 2022). A general lack of standardized testing protocols for measuring zebrafish behavioral performance may contribute to those intra- and interlaboratory differences (Gerlai, 2019; Hsieh et al., 2022).

Many of the protocols used in larval zebrafish behavioral assays are unique to each individual laboratory. These various larval zebrafish developmental and neurodevelopmental methods have been reviewed herein, with a focus on subsets of variables that could influence the toxicity assessment outcome (Hamm et al., 2018; Legradi et al., 2015; Ogungbemi et al., 2019). While those reviews highlighted differences in developmental zebrafish protocols, they also called for increased reporting of detailed experimental parameters. Over the last decade, reporting guidelines and recommendations for general animal research and ecotoxicology have been published (e.g., Animals in Research: Reporting In Vivo Experiments [ARRIVE] and Criteria for Reporting and Evaluating Ecotoxicity Data [CRED] [Kilkenny et al., 2010; Moermond et al., 2016; Percie du Sert et al., 2020]). Additionally, some journals are now requiring adherence to reporting guidelines for publication, but it is not clear to what extent these guidelines and standards have been implemented or enforced. While some aspects of zebrafish experiments are approaching standardization, such as rearing and husbandry protocols, (Westerfield, 2007; Council NR, 2011; Cartner et al., 2019; Aleström et al., 2020), there are currently no standard guidelines consistently followed for either conducting or reporting zebrafish behavior-based research despite the increased call for transparency.

The present study employs a literature review to assess the status of method variable reporting for developmental neurotoxicity assays using larval zebrafish behavior. We focused on the following areas: larval zebrafish aged 5 to 7 days post fertilization (dpf), developmental exposure to any of the 61 chemicals that were previously investigated by our laboratory for developmental neurotoxicity, chemical effects of the treated larvae in comparison to the control, and behavioral outcomes in response to light/dark photoperiod transitions. These parameters were chosen to focus the review on assays similar to those utilized by our laboratory (Jarema et al., 2022). Here we screened abstracts and full text articles, extracted information on an extensive list of method variables, and performed a risk-of-bias evaluation (Hooijmans et al., 2014). To our knowledge, this is the first time such an approach has been applied to this research area. Our main goals were to identify gaps in information, assess inconsistencies in reported methods, and recommend reporting guidelines for larval zebrafish behavioral research.

2. Methods

2.1. Approach

Publications of larval zebrafish high-throughput screening approaches are on the rise but seldom are the results of individual laboratory testing compared to published results gathered using a literature review approach. The main goals of our review were two-fold: (1) gather published screening results from the same chemicals for comparison to our laboratory chemical screening results (previously published, Jarema et al., 2022) and (2) gather method information to determine if methods were consistent by reviewing the reporting of method variables (presented herein). Our approach was to include only explicitly stated information in the publications, primarily housed in the Materials and Methods sections, because if an experiment was to be repeated, very minimal or no interpretation or assumptions should be taken. Therefore, our summary of parameter information was either reporting the explicitly stated parameter or an indication of “Not stated.” If only partial information was provided in the publication, then a “Not stated” indication was given, since it was not explicitly stated/clear what was completed. The following literature review was conducted by six experienced researchers with knowledge in larval zebrafish behavior assays.

2.2. Searching strategies

A search of the literature was conducted to gather articles that reported methods used for investigating the neurodevelopmental effects of chemicals in zebrafish assays (Fig. 1). Our approach was to focus on publications that utilized a larval zebrafish light/dark locomotor assay to serve as a complement to a screening effort by our laboratory (Jarema et al., 2022). Here we highlight the method information for those publications that were used to compare results.

Fig. 1.

Overview of the literature identification process. Publications were first gathered using the Abstract Sifter (Baker et al., 2017) using chemical name or CAS registry number with broad and specific search terms. Level 1 screening included title and abstract review by a team of three reviewers (1 reviewer per abstract) focusing on larval zebrafish behavior assays. Level 2 screening involved full text review by two independent reviewers per text, with inclusion of publications for this screening level determined using the criteria in the order listed. For example, each paper that was gathered for full text review was first screened to confirm that the chemical was introduced within the first 3 days post fertilization (dpf). If it was explicitly mentioned that the chemical was introduced outside this window (e.g., 5 dpf) then that publication was removed. Next, the remaining publications were screened for chemical duration, using the same exclusion/inclusion approach, followed by behavior testing age, behavior protocol, and acclimation phase. Any publication that did not explicitly meet one or more of these criteria was excluded.

To begin, abstracts were gathered, between June and September 2020, using the Abstract Sifter (Baker et al., 2017), a program that collates abstracts from the PubMed database (see Supplemental Table 1.1–1.2 for search results). The following search terms were used to gather abstracts with no time filters/restrictions:

• “chemical name OR CAS” and “zebrafish AND neurotox*,” hits = 2,77

• “chemical name OR CAS” and “zebrafish AND (behavior OR locomotor) AND develop*,” hits = 116

• “chemical name OR CAS” and “zebrafish AND (swim OR swimming OR locomotion),” hits = 362

• “chemical name OR CAS” and “zebrafish,” hits = 2,411

2.3. Literature screening and inclusion criteria

2.3.1. Level 1 screening –title and abstract

All abstracts gathered via the Abstract Sifter were divided and reviewed among three reviewers. Abstracts that mentioned utilizing a larval zebrafish light/dark behavioral assay were considered relevant, and their corresponding texts were moved to Level 2 screening. Any publication in which a firm decision could not be made by the single reviewer was included for full text review (Level 2 screening).

2.3.2. Level 2 screening – Full text

For the Level 2 screening, each full text was reviewed by two independent reviewers using the inclusion criteria noted below (see Supplemental Table 1.3 for full text publications reviewed). We developed these criteria to accomplish our goal of targeting larval behavioral light/dark transition assays where a developmental chemical exposure occurred.

2.3.3. Inclusion criteria

• The chemical was introduced to early life stage zebrafish (Danio rerio of any strain) between 0 and 3 days post fertilization (dpf).

• The chemical exposure duration was at least 24 h.

• The behavioral test took place between 5 and 7 dpf.

• The behavior protocol included at least one light to dark photoperiod transition.

• There was an acclimation phase, of any duration, immediately prior to behavior testing.

Publications were screened using the criteria in the order listed above and any publication that did not meet one or more of the above criteria was excluded. For example, each paper that was gathered for full text review was first screened to confirm that the chemical was introduced within the first 3 days post fertilization (dpf). If it was explicitly mentioned that the chemical was introduced outside this window (e.g., 5 dpf) that publication was removed. Next, following the same exclusion/inclusion approach, the remaining publications were screened for exposure duration, then behavior testing age, followed by behavior protocol, and acclimation phase. If a cohesive decision could not be made between the two reviewers, then the publication in question was discussed among the entire group of reviewers (6 people) until a consensus was reached.

2.4. Data extraction and collection

Specific information on 51 unique method variables describing chemical information, adult husbandry, rearing conditions, behavior assessment, and statistical analysis was extracted into a Microsoft Excel spreadsheet from the included publications (Supplemental Table 2.1, 2.2). These variables were selected due to their potential to influence zebrafish behavior results (Legradi et al., 2015) in addition to those determined by the zebrafish expert group would be needed to repeat an experiment accurately and precisely. Some of these variables may be related to each other. We included these separately to evaluate the transparency of reported methods as these details may independently influence the toxicity outcome. For example, the answer to the question of did the authors exclude larvae with uninflated swim bladders from behavior analysis? could be the exact same for the question did the authors exclude larvae from behavior analysis? based on the assessment criteria outlined by the publication authors. Since swim bladder inflation has been shown to influence behavior in the absence of other more severe morphological malformations (Hill et al., 2021) we elected to keep this a separate method variable.

The same two independent reviewers that screened each paper for relevance during Level 2 screening also independently extracted the variable information from each publication, then discussed findings and came to an agreement for one cohesive entry (Supplemental Table 2.2). Again, any discrepancies were discussed among the total group of reviewers to ensure that only information explicitly stated in the article was included and not interpretations of methods by reviewers. Publication, chemical, and variable information were recorded in a Microsoft Excel spreadsheet which could then be sorted and analyzed to determine consistencies and missing information among the publications (Supplemental Table 2.2, 2.3).

2.5. Statistics and data analysis

Since one of our main goals was to survey the overall status of variable reporting and the specific parameters that are reported for each variable, we calculated three main endpoints. First, to determine the occurrence of parameters reported for a specific variable, the number of times the same parameter was reported was divided by the total number of publications that explicitly stated that variable. If a publication reported a range of values, such as temperature reported as 28–29 °C, the average of the range was used for subsequent calculations (e.g., 28.5 °C). Second, to determine if the percentage of variable reporting has increased with the publication of reporting criteria and guidelines, the relationship between overall reporting percentage (number of variables reported divided by the total number of variables targeted [51]) and year of publication was plotted as a linear regression to determine any trend. Lastly, to determine if there was consistency among the parameters listed for each variable, the number of different responses reported for each variable were counted and plotted against the overall reporting percentage. The overall reporting percentage was determined by dividing the number of publications that explicitly reported a variable by the total number of publications (n = 31). For variables that were phrased as a question, such as was a negative control included?, the number of different answers to that question were used as the number of parameters. Linear regression was generated using SigmaPlot 14. Pie charts and bubble plot were generated using R for Windows Version 4.0.3.

2.6. Risk of bias analysis

To further assess other aspects of reporting quality, we applied a risk of bias analysis to each publication based on guidelines for clinical animal research (SYRCLE [Hooijmans et al., 2014]) and for zebrafish fish embryo toxicity test (FET) review (Stephens et al., 2018). The risk of bias tool aids in assessing the quality of information from each publication focusing on study design, randomization, and experimental “blindness” (or prevention of operator bias). Basing our approach on Stephens et al. (2018), we included the same ten risk of bias questions (Hooijmans et al., 2014) and three reporting criteria (Krauth et al., 2014) (Table 1). Like our literature review, two independent reviewers would each assess a publication, assign a “yes,” “no,” or “unknown” answer to each risk of bias question, and come to an agreement on a unified response. In each case (e.g., Were the experiments blinded?) a “yes” answer indicates a low risk of bias, “no” indicates a high risk of bias and, “unknown” indicates that there is not enough information provided to assess the level of risk of bias (same approach taken for the “Not stated” classification mentioned above). See Supplemental Table 3 for the protocol developed to specifically address risk of bias in larval zebrafish behavior assays.

Table 1.

Risk of bias analysis results for all publications included in the review. Questions based on risk of bias analysis conducted by Stephens et al., 2018 with tools developed by Hooijmans et al., 2014 and Krauth et al., 2014. Low risk of bias indicated by blue (yes), high risk of bias indicated by red (no), and unclear risk of bias indicated by grey (unknown, not explicitly stated). White spaces indicate that the question was not applicable. Answers were based on information that was explicitly stated to avoid interpretation errors. Publication identifications are blinded but the bibliography can be found in Supplemental Table 2.3. ^aEach column represents information for an indiviual publication.

3. Results

3.1. Search results and overall variable reporting

The literature search generated 2,367 abstracts which were narrowed down to 94 publications based on Level 1 screening regarding relevance to zebrafish larval behavioral assays (Fig. 1). From these abstracts, Level 2 screening resulted in 31 publications, published between the years 2011 to 2020, that covered 36 chemicals (N.B., some publications tested more than one chemical; see Supplemental Table 2.3 for included publication information). Only values that were explicitly stated by the authors of each publication were included in this review. Some challenges were encountered due to inconsistent terminology when the authors described their methods. For this reason, we opted to phrase some of the targeted variables as questions instead of using a variable name. For variables that were not explicitly stated or unclear, a “Not stated” entry was reported and are represented as the “Not stated” category. Also, there were some variables that were “Not applicable.” For example, if it was explicitly stated that a publication did not dechorionate embryos then the answer to the question for those that reported dechorionating, what was the method? was “not applicable”. If it was not explicitly stated that embryos were or were not dechorionated, then “Not stated” was entered for all related variables.

Overall, there was not a single publication that reported all the methodological variables targeted in this effort. Reporting of explicitly stated variables spanned from 51 to 86% (Fig. 2). Eighteen out of the 31 included publications had an overall reporting percentage of <75%. To determine if there was an increase in variable reporting following the publication of reporting guidelines such as ARRIVE and CRED (Kilkenny et al., 2010; Moermond et al., 2016; Percie du Sert et al., 2020), beginning in 2010, a comparison of the overall reporting percentage each year was conducted. As shown in Fig. 2, there was not a significant relationship between year published and reporting percentage (slope parameter equal to −0.251; CI: −0.251 ± 1.22; p = 0.677), so it appears there has not been an increase in variable reporting since the publication of those guidelines.

Fig. 2.

Overall variable reporting for each of the publications included in the review (n = 31) versus the year of publication. Variable reporting was calculated for each publication by dividing the number of variables explicitly stated over the total number of variables. Vertical lines represent the year of publication of ARRIVE guidelines (red line; [Kilkenny et al., 2010]), CRED guidelines (green line; [Moermond et al., 2016]), and Legradi et al., 2015 zebrafish behavioral review paper (blue line). Linear regression of year to overall variable reporting was applied and no trend in the data was observed highlighting that there has been no increase in variable reporting after the publications of these guidelines/criteria (slope parameter equal to −0.251; CI: −0.251 ± 1.22; p = 0.677). All publications were included in the linear regression. There were some instances where multiple publications had the same overall variable reporting percentage published in the same year leading to an overlap in data points.

3.2. Reporting per variable and variable group

In Fig. 3, the x axis represents the percentage of the publications explicitly reporting the parameter for the given variable. There were 15 variables that had the highest (100%) reporting percentage including: age when chemical is administered, age when chemical exposure ends, number of wells during rearing, experimental rearing venue, estimated chemical exposure duration, behavior apparatus, number of concentrations tested for behavior, age at time of behavior testing, total duration of behavior protocol and number of photoperiod changes. Additional variables with the highest reporting percentage include, were embryos group or single housed?, for those that reported dechorionating, what was the method?, was a negative control included?, for those that stated using a positive control, what was the chemical used? and how were differences in behavior determined? The variables least reported were acclimation period light intensity (10%), was a positive control included? (10%) and were data transformed? (10%), followed by were outliers identified (13%) and did the authors exclude larvae with uninflated swim bladders from behavior analysis? (16%). Again, it should be noted that some variables, such as acclimation period light intensity and did the authors exclude larvae with uninflated swim bladders from behavior analysis? may be related to other variables targeted (behavior acclimation photoperiod and did the authors exclude larvae from behavior analysis?) but were included separately to better evaluate method transparency. Dividing variables into 4 methodological subgroups (i.e., Chemical information and Adult husbandry, Rearing, Behavior, and Statistics), the Rearing category had the highest level of reporting with an average of 90% of these variables explicitly stated. In contrast, the lowest reported category was Statistics which only included, on average, 66% of targeted variables.

Fig. 3.

Parameter and reporting percentages for all method variables targeted for review. The reporting percentage for each variable (depicted on x axis) was determined by the number of publications that explicitly stated each variable, divided by the total number of publications (n = 31). Circle size represents the number of parameters or answers to questions for each variable. These numbers are also in paratheses next to variable name listed on the y axis. Triangles represent variables with one parameter reported. Shading represents variable category. Higher consensus and reporting for parameters/variables are represented as smaller circles located on the far right of figure, whereas larger circles indicate inconsistencies with reported parameters. For example, media composition had 8 different parameters reported and was reported in roughly 75% of publications. Compare this variable to behavior apparatus, which only had two different parameters reported and was reported in 100% of publications. * “Did the authors exclude larvae from behavior analysis?” - does not include swim bladder inflation. Refer to Supplemental Fig. 1 for full detail of entries.

3.3. Inconsistencies of parameters reported

Similar to the overall reporting percentages of variables among publications (x-axis of Fig. 3), the reporting of parameters was mixed. Figs. 3 and 4 and Supplemental Fig. 1 visualize the inconsistencies in response to each of the variables analyzed in this literature review. This is depicted in two places: the first is the size of the circle for each variable in Fig. 3, and then in Fig. 4 and Supplemental Fig. 1 where each pie chart shows the reported parameter for each variable with the size of each slice representing the percent of publications reporting that particular parameter. Fifty-one unique pie charts are available describing the heterogeneity in reported results across each experimental variable (Fig. 4, Panels A-D and Supplemental Fig. 1, Panels A through AX). It should be noted that neither the pie slices for a specific parameter and the size of circles consider the “Not stated” portion of those variables (Supplemental Fig. 1, grey “Not stated” slices). The inconsistencies within these variables are highlighted by reporting the number of different parameters (size of circles in Fig. 3).

Fig. 4.

Subset of pie charts illustrating parameters reported per method variable. To determine the frequency of parameters reported for a specific variable (size of pie slices), the number of times the same experimental parameter was reported was divided by the total number of entries. If a publication reported a range of values, such as temperature reported as 28 to 29 °C, the average was used for subsequent calculations. “Not stated” represents variables that were not explicitly reported in publications. Refer to Supplemental Fig. 1 or full details of every variable (n = 51) targeted (selected pie charts are Panels B, P, AD, U respectively).

The more inconsistent the choice of parameters are for a given variable, the larger the circle denoting that there were a higher number of different parameters reported. For example, Fig. 3, depicts media as one of the larger circles representing the various types utilized in the selected publications (11 different types reported). This is confirmed by looking at the many different sized pie slices located in Supplemental Fig. 1, Panel F. We also found that for some variables such as media (11 parameters reported) and media composition (8 values reported, Supplemental Fig. 1, Panel G) there were multiple media types for the same media composition and further the same salt concentration was reported but using three different units. Other variables that had a high number of different parameters reported include for the reported solvent (vehicle), what was the final percent (%)? (n = 13, ranging from 0.001 to 0.64%, Fig. 4, Panel A), volume in each rearing unit (n = 14, ranging from 100 to 500,000 μL, Supplemental Figure, Panel T) and total duration of behavior protocol (n = 16, ranging from 8 to 70 min, Fig. 4, Panel C).

Now focusing solely on variables with only one reported parameter, these include type of behavior venue (e.g., parameter = microtiter plate) and number of larvae per behavior well (e.g., parameter = one), for the Behavior subgroup (red shaded region in Fig. 3 and Supplemental Fig. 1, Panels Y and AG). Other variables that had only one reported parameter included for those that reported dechorionating, what was the method? (enzymatically), (Rearing subgroup, orange shaded region in Fig. 3 and Supplemental Fig. 1, Panel J), was a negative control included? (yes), for those that stated using a positive control, what was the chemical used? (chlorpyrifos), did the authors exclude larvae from behavior analysis? (yes), were data excluded? (yes), were all experimental conditions represented on each plate? (yes), were outliers identified? (yes), was a positive control included? (yes), and were data transformed? (yes), (Statistics subgroup, green shaded region in Fig. 3 and Supplemental Fig. 1, Panels AM, AN, AS, AT, AU, AX, AW). While these variables appear to exhibit similar patterns with the number of parameters selected/reporting, it should be noted that not all publications explicitly stated their choice for each variable (i.e., the “not stated” category) with reporting percentages for variables falling between 10% and 100%. This lack of reporting can mask the true diversity of these experimental practices. The parameters for each variable reported by each publication and the incidence of these values reported among all publications for each variable can be viewed in Supplemental Fig. 1.

To determine the level of consistency of parameters reported for each variable, we counted the number of different parameters reported among methods. Panels A-AY of Supplemental Fig. 1 show that number of embryos per rearing unit (10 different variables ranging from 1 to 500 embryos; Supplemental Fig. 1, Panel S) and volume in each rearing unit (14 different variables ranging from 100 to 500,000 μL; Supplemental Fig. 1, Panel T) were similar in the representation of each parameter, compared to the age when chemical was administered that had three primary ages reported (among ten different parameters in total, spanning 2 to 48 hpf; Fig. 4, Panel B). Other variables that resulted in more unequal representation included incubator temperature during rearing (°C) with five values ranging from 26 to 29 °C (Fig. 4, Panel D) and reported in 74% publications (Fig. 3), and temperature of the testing room (°C) with three values ranging from 26 to 28.5 °C (Supplemental Fig. 1, Panel AJ) and reported in 32% publications (Fig. 3).

3.4. Risk of bias analysis

To evaluate the reporting quality of information in terms of randomization and experimental blindness, a risk of bias analysis was conducted on all publications included in the review. As shown in Table 1, there were no publications that had ideal reporting for all experimental aspects (as indicated by at least one red symbol in each column), with most publications not reporting enough information, leading to an “unknown” entry for most questions asked. Some challenges faced with the risk of bias assessments included criteria that were too general, or it was not clear how these applied specifically to zebrafish assays. For example, the question “were animals selected at random for outcome assessment?” often did not apply because embryo/larvae exposures may occur in the same venue as behavioral testing (Hooijmans et al., 2014). To make this more approachable, we elected to reword these questions to be better suited for the specific assay selected for this review (Table 1, Supplemental Table 3). The highest risk of bias observed was for the question was a power/sample size shown? followed by were the experiments blinded and were the experiments randomized? (Table 1). Only 55% of publications explicitly mentioned randomization, only 9% reported that the experiment was blinded, and there were no publications that reported using a power or sample size calculation. We included these since randomization has been incorporated into journal best publishing practices and reporting guidelines mentioned previously (e.g. ARRIVE and CRED) and power/sample size calculations can be useful for decreasing overall numbers of required larvae which is aligned with the 3Rs principles: replace, reduce, and refine and institutional goals (Russell and Burch, 1959; Krewski et al., 2010; USEPA, 2019; USEPA, 2020). The lowest risk of bias was for the question was the study protocol and all of the pre-specified outcomes reported in the manuscript? which focused on having a method (albeit not incorporating details) described for each result reported in the publication.

4. Discussion

The conservation of nervous system development among vertebrates and its control of behavior makes larval zebrafish behavior assays of particular interest for developmental neurotoxicity (DNT) testing. Developmental zebrafish assays provide large amounts of data relatively quickly and are being utilized for higher throughput testing in the DNT realm (Basnet et al., 2019; de Esch et al., 2012a; Guo, 2009; Rihel and Schier, 2012). Many different assays with different method variables have been chosen to investigate similar behavioral assays/endpoints including the light/dark transition test (Legradi et al., 2015). The main goal of this paper was to evaluate the reporting status of an extensive list of method variables spanning from embryo preparation to statistical methods. The focus of this literature review was specifically on behavioral assays that utilize light/dark transitions. Although it only concentrates on a narrow window of this research area, we found many lessons to be learned on the status of reporting method variables.

To our knowledge, this is the first time this type of review has been applied to zebrafish behavioral methods focusing on an extensive list of 51 method variables. The first lesson learned was that there are some methodological variables that are routinely reported; approximately 30% of our targeted variables had 100% reporting, such as age when chemical is administered, age when chemical exposure ends, number of wells during rearing, experimental rearing venue, number of concentrations tested for behavior, and age at time of behavior testing. This may be due to our inclusion criteria which were specifically designed to incorporate assays that were similar to the light/dark transition assay utilized by our laboratory to test a chemical set for DNT (Jarema et al., 2022). However, while many of our targeted variables had an overall high (>75%) reporting frequency, there were many possible options for each variable that could lead to challenges with data interpretation. Of note, two variables within the least reported variable category (Statistics), how were differences in activity determined? and behavior endpoint, both had the third highest parameter count (n = 13 different statistical tests and locomotor behavior/activity endpoints), further showcasing those differences in how data are collected and analyzed. In addition to the inconsistencies in reported parameters, there were also different terminologies and units used, such as reporting media composition, making initial comparisons difficult. The second lesson learned was that, despite the publication of reporting guidelines/criteria and extensive zebrafish assay review papers, there remains a deficit in variable reporting (Kilkenny et al., 2010; Legradi et al., 2015; Moermond et al., 2016; Percie du Sert et al., 2020). Expansion of this effort to include information related to the variables described here when reporting zebrafish behavioral results will further heighten our understanding within the greater zebrafish behavioral research community and allow us to better apply knowledge gained from zebrafish experiments.

Risk of bias evaluations focus on various aspects of the method: for example, animal handling or evaluation in a random, blinded manner. While there are many tools available (Krauth et al., 2013), our risk of bias approach was based on one recently applied to zebrafish embryotoxicity assays to determine the ability of test method performance to predict mammalian toxicity and feasibility of applying these approaches in this context (Stephens et al., 2018). Like previous assessments that applied a risk of bias tool to zebrafish developmental assays, there was a large amount of unknown or unclear risk of bias stemming from the lack of reporting quality (Stephens et al., 2018). An area of difficulty involved the risk of bias question focusing on adjusting for confounders and incomplete data in terms of methods reporting quality because these have not been well defined for zebrafish experiments. In the present review, we addressed this by focusing solely on the relationship between developmental toxicity (malformations and/or mortality) and behavior (excluding developmental toxicity). As noted above, other challenges that arose were the general wording of the questions, and clarity on how those applied specifically to zebrafish assays (e.g., targeted towards traditional mammalian animal models). For these reasons, we opted to reword these questions to better represent our interpretation which we also hope will increase the application of these tools to zebrafish research (Table 1 and Supplemental Table 3). Additionally, through using the risk of bias tool, we found a lack of reporting proper data quality assurance/quality control methods, including the reporting of missing data not related to chemically induced effects such as malformations or mortality.

Intra- and inter-laboratory variability is frequently observed for larval zebrafish behavioral data, and the differences in assay protocols used by different laboratories may contribute to these patterns (Gerlai, 2019; Hsieh et al., 2022). There are many different factors that contribute to a given experimental protocol and some variables have been shown to affect larval zebrafish. For example, light intensity selected for both light and dark photoperiods has been shown to influence the behavioral pattern (Hartmann et al., 2018; MacPhail et al., 2009; Padilla et al., 2011). In the current review, there were seven different values selected for the light photoperiod intensity, which makes comparing study results difficult; most publications (68%) did not even include this information. Other examples of variables that affect basal larval zebrafish activity or influence chemically induced behavior changes are fish strain (de Esch et al., 2012b; Lange et al., 2013; van den Bos et al., 2017), age at time of behavior testing (de Esch et al., 2012b; Fraser et al., 2017; Ingebretson and Masino, 2013; Padilla et al., 2011; Selderslaghs et al., 2010), light:dark cycle during rearing (Hurd and Cahill, 2002; Wilson et al., 2020), chorion status (Wilson et al., 2020), temperature of behavior testing room (Abozaid et al., 2020), number of wells and volume in wells for behavior testing (Ingebretson and Masino, 2013; Padilla et al., 2011), photoperiod protocol (MacPhail et al., 2009), and excluding malformed larvae (Padilla et al., 2011).

While standardized protocols will make assay and data comparison easier, it is important that consistent and comprehensive reporting of experimental variables occurs before standardized protocols can be developed. Building on previous efforts (Kilkenny et al., 2010; Legradi et al., 2015; Moermond et al., 2016; Percie du Sert et al., 2020), we present a novel, extensive list of reporting guidelines, the Developmental Assays for Zebrafish Larvae Guidelines (DAZL, Table 2). The focus of these guidelines is to provide a checklist when planning, executing, analyzing, and reporting zebrafish developmental and behavioral research. Using best practices (e.g., including an assay positive control) and describing these aspects in detail will enable an improved comparison of results and methods. Additionally, there should be increased responsibility of journals to require these parameters in publications as standardized reporting will improve overall reporting quality in this field.

Table 2.

Developmental Assays for Zebrafish Larvae (DAZL) guidelines for comprehensive reporting of methodological variables for developmental and behavioral larval zebrafish research. Purpose of this table is to serve as a recommended checklist when writing protocols and research articles.

I. Environmental and Husbandry Conditions:

Identification:

◽ Indicate strain, age, and source of breeding stock and embryos, include age during testing.

Housing:

◽ Describe lighting conditions (light/dark hour cycles, light intensity), water quality parameters (e.g. pH, dissolved oxygen, ammonia), temperature, feeding/breeding schedules (e.g. type of food and frequency of feeding), and housing rack information for breeding stock.

◽ Describe embryos/larvae rearing conditions including light cycle, media, temperature, feeding, incubator information/settings before and during experiments. State if embryos/larvae were randomly housed.

◽ For embryos/larvae, describe type of plate/container, including number per well/container and liquid volumes. State if embryos remained in the same experimental venue or if they were transferred to a different testing venue. If so, include whether the embryos/larvae were randomly distributed.

Embryo Preparation:

◽ Describe how embryos were washed, and any pre-exposure procedures.

◽ State if embryos were dechorionated, and if so, describe dechorionation procedure.

II. General Experimental Conditions:

Exposure:

◽ For all experimental chemicals, vehicle, and positive/negative controls, list the CAS Registry Number or EPA's DSSTox ID, describe chemical exposure procedure during all experimental conditions including age at initial exposure, concentrations, volume per well, exposure duration, chemical renewal/depuration (include age and time of day), and if/how embryos/larvae were randomly distributed into treatment groups.

◽ Describe stock plate with chemical concentration arrangement, noting plate size, number of wells, all chemical concentrations, positive/negative controls, and blinding/randomization procedures.

◽ Provide a schematic of experimental plate with chemical concentrations identified, if possible.

Assessment:

◽ Describe morphology evaluation/assessment procedure during rearing including equipment used, embryos/larvae age, time of day, frequency of assessments, evaluation criteria, number of assessors, and if assessors were blinded to treatment groups. Optional, include a copy of morphology assessment sheet..

Disposition of test animals:

◽ Describe euthanasia procedure, include age and time after final experimental conditions.

III. Behavioral Testing Conditions:

Pre-testing:

◽ Describe all procedures/conditions prior to behavioral testing, including timing of plate changes/transfers (i.e. days within experiment), embryos/larvae assessment, and relocation to testing room.

◽ Describe room acclimation period prior to testing, include location (testing room, behavior apparatus, or incubator), lighting, temperature, and sound

Testing:

◽ Describe if the chemical is present in solution during testing or if it was removed. If removed, state how long the depuration occurred.

◽ Describe behavioral testing room (lighting and temperature), testing apparatus, and recording/tracking equipment (brand and model).

◽ Explain behavioral testing schedule and procedure, including acclimation period, light/dark transitions, total session time, light level (lux), start/end times, temperature, and other stimuli.

Post-testing:

◽ Describe post-testing procedures including larvae assessment/evaluation and imaging. Include evaluation parameters, number of assessors, and if assessors were blinded to experiment.

◽ Provide detailed imaging procedure, if applicable

Data inclusion/exclusion:

◽ Report any adverse/unanticipated effects that resulted in data exclusion such as experimental/tracking errors and equipment issues.

◽ Describe criteria for data inclusion/exclusion from analyses, such as dead, unhatched, malformations, and uninflated swim bladder. Explain the rationale for exclusion of animals, if applicable.

◽ Describe criteria for test-concentration or control group inclusion/exclusion. If a percentage cutoff was used as inclusion criteria, state the cutoff and explain the rationale.

Data acquisition:

◽ Describe video tracking/analysis procedure, state all equipment and software used.

◽ State which outcomes/variables were included in analyses.

IV. Study Design and Data Analysis:

Sample Size:

◽ Explain criteria for determining sample size (e.g., a priori sample size calculation), stating whether revisions were made after initial data collection.

◽ Report final sample size for each experimental/analysis group, noting excluded data.

Incomplete Data:

◽ Report any incomplete behavioral tracking included and state whether adjustments were made in statistical analyses.

Statistical Methods and Results:

◽ List all software used, name and version.

◽ Describe data transformation procedure/rationale. Describe metrics used and calculation procedures where applicable. State whether transformation decisions were made before or after initial data collection.

◽ State whether outlier analysis/removal was performed, and if established before or after initial data collection.

◽ Describe any statistical test performed and state whether the chosen test examined the same larvae overtime (e.g., a repeated-measures analysis).

◽ Report descriptive statistics for each group, with a measure of variability where applicable.

Data Access:

◽ State whether experimental data and analysis scripts/code are publicly available and include URL if available.

5. Conclusions

In conclusion, we surveyed data and produced reporting guidelines on 51 method variables necessary to reduce intra- and inter-laboratory variability for repeating experimental zebrafish behavioral assays. Inconsistencies observed in these data demonstrate a need for increased transparency and reporting of experimental variables to enable researchers to understand the details and repeat experiments without interpretation/assumptions. Efforts have been made to develop a harmonized protocol for zebrafish assays, but these have not included behavior as an endpoint (Ball et al., 2014; Gustafson et al., 2012). Currently, there is a need for a robust study comparing multiple different variables simultaneously to decipher which variables control the high variability commonly observed; these unknowns need to be determined prior to the development of a harmonized protocol. Additionally, consistency will increase reproducibility of results. Although there were some challenges in applying a literature review and risk of bias tool to a narrow area of zebrafish behavioral research, these approaches enabled us to collect information in a controlled way for a better understanding of the similarities and differences, both in the overall variable reporting and number of parameters reported, in developmental larval zebrafish light/dark transition assays among laboratories. We hope that the conclusions drawn from this process will enable conversations among zebrafish researchers and lead to more consensus regarding the highlighted experimental variables.

Data availability

Data collected from publications are available in SI Table 2.