Reporting quality in preclinical animal experimental research in 2009 and 2018: A nationwide systematic investigation

Birgitte S Kousholt; Kirstine F Præstegaard; Jennifer C Stone; Anders Fick Thomsen; Thea Thougaard Johansen; Merel Ritskes-Hoitinga; Gregers Wegener

doi:10.1371/journal.pone.0275962

. 2022 Nov 3;17(11):e0275962. doi: 10.1371/journal.pone.0275962

Reporting quality in preclinical animal experimental research in 2009 and 2018: A nationwide systematic investigation

Birgitte S Kousholt ^1,^*, Kirstine F Præstegaard ¹, Jennifer C Stone ², Anders Fick Thomsen ¹, Thea Thougaard Johansen ¹, Merel Ritskes-Hoitinga ^1,³, Gregers Wegener ^1,⁴

Editor: Gillian Currie⁵

PMCID: PMC9632797 PMID: 36327216

Abstract

Lack of translation and irreproducibility challenge preclinical animal research. Insufficient reporting methodologies to safeguard study quality is part of the reason. This nationwide study investigates the reporting prevalence of these methodologies and scrutinizes the reported information’s level of detail. Publications were from two time periods to convey any reporting progress and had at least one author affiliated to a Danish University. We retrieved all relevant animal experimental studies using a predefined research protocol and a systematic search. A random sampling of 250 studies from 2009 and 2018 led to 500 publications in total. Reporting of measures known to impact study results estimates were assessed. Part I discloses a simplified two-level scoring “yes/no” to identify the presence of reporting. Part II demonstrates an additional three-level scoring to analyze the reported information’s level of detail. Overall reporting prevalence is low, although minor improvements are noted. Reporting of randomization increased from 24.0% in 2009 to 40.8% in 2018, blinded experiment conduct from 2.4% to 4.4%, blinded outcome assessment from 23.6% to 38.0%, and sample size calculation from 3.2% to 14.0%. Poor reporting of details is striking with reporting of the random allocation method to groups being only 1.2% in 2009 and 6.0% in 2018. Reporting of sample size calculation method was 2.4% in 2009 and 7.6% in 2018. Only conflict-of-interest statements reporting increased from 37.6% in 2009 to 90.4%. Measures safeguarding study quality are poorly reported in publications affiliated with Danish research institutions. Only a modest improvement was noted during the period 2009–2018, and the lack of details urgently prompts institutional strategies to accelerate this. We suggest thorough teaching in designing, conducting and reporting animal studies. Education in systematic review methodology should be implemented in this training and will increase motivation and behavior working towards quality improvements in science.

Introduction

Poor reproducibility and translational failure in biomedical research lead to skepticism regarding the reliability of preclinical research findings. The reasons are multi-factorial [1–4]. A prevalent issue is unsatisfactory internal validity. Internal validity is the extent to which a design and conduct of a study eliminates the possibility of systematic errors (bias) [4]. Appropriate methodologies safeguarding against systematic errors can be implemented in the design, conduct, and analysis of an experiment in order to increase the internal validity [4].

Essential safeguards are blinding, randomization, and a thorough description of animals’ and samples’ flow including reasons for exclusion [5]. The judgment of the scientific evidence is hampered if these measures are poorly reported [6]. Evidence exists that lack of reporting corresponds to the absence of conduct [7, 8]. Systematic reviews of preclinical animal studies disclose smaller effect sizes when randomization and blinding are implemented compared with studies not reporting these precautions [9–12]. This finding is corroborated in meta-epidemiological studies of clinical data that identify a negative additive impact when more than one safeguard is omitted [13–16]. Attrition bias (i.e., poor handling of dropouts) skews data and jeopardizes a study’s scientific robustness. Holman et al. demonstrate that losses of only a few animals in a study can distort study effects [17]. A safeguard of importance is how the sample size is reached. Preclinical animal experiments often carry (too) small group sizes. A drawback to this is that positive findings may be due to chance rather than actual effect [18–20]. Thus, comprehensive sample size calculations based on the best available evidence are paramount. Other influential quality factors are, for example, the animals’ health status or comorbidities before and during experiments, as undetected diseases may affect the study outcome [21–23].

The Animal Research: Reporting of In Vivo Experiments (ARRIVE) guidelines for animal experiments reporting, first published in 2010, provide recommendations on improving low reporting standards and have recently been updated [24, 25]. However, studies repeatedly show inadequate reporting of quality indicators [26–30], suggesting the unsuccessful implementation of the guidelines, even though over 1000 journals endorse them. The implementation may be hindered by the lack of engagement of multiple stakeholders who all must engage in improving the reporting quality. In this context, the use of the ARRIVE guideline by researchers is necessary already at the planning stage to help improve experimental design and, in turn, improve reporting. Previous research has investigated the prevalence of reporting of measures to reduce the risk of bias for specific animal disease models or subjects of interest [28–31]. Other previous evaluations of preclinical reporting have provided an overview of the reporting status of items related to the internal validity or rigor of these experiments (e.g. blinding and randomization) [32]. This study investigates the reported information’s level of detail by assessing preclinical studies within all animal experimental research fields with one or more authors affiliated with Danish research institutions. In part I of the study, we focus on the overall reporting status of methodological safeguards. In part II, the focus is on the level of detail given for each reported item. To detect whether progress over the years exists, we investigated publications containing experiments published before (the year 2009) and after (the year 2018) publication of the ARRIVE guidelines [24].

Materials and methods

The experimental design was based on random sampling to avoid bias. An equal number of studies from each year were included to compare the results between the two time periods. It was estimated that a thorough assessment of 500 papers in total– 250 papers from each year–could be performed within the given timeframe.

Study protocol

To further prevent methodological flaws and minimize bias, we modified a pre-specified systematic review protocol for animal intervention studies offered by the SYstematic Review Centre for Laboratory animal Experimentation (SYRCLE) [33]. The protocol was uploaded at SYRCLE (7^th of November 2018) for guidance and feedback and is found in the supporting information (S1 File).

Selection of studies

In collaboration with a library information specialist, we retrieved all potentially relevant studies using a modified, comprehensive search strategy [34, 35]. The search was systematically performed in two databases, Medline (via PubMed) and Embase. All in vivo studies conducted in non-human vertebrates with one or more authors affiliated with at least one of five Danish universities of interest were retrieved. The search was divided into two separate searches based on publication year (search 1: 1^st of January 2018 until 6th November 2018; search 2: the year 2009). The studies were imported to two dedicated EndNote libraries (EndnoteX8, Clarivate Analytics, Philadelphia, USA), and duplicates were removed. One thousand, one hundred and sixty-one studies from 2009 and 1890 studies from 2018 were found.

The information from the Endnote libraries was copied to Excel (MS Office, version 2016, Microsoft Corp., USA), and the publications from each year were randomized using the “= RAND()” command, thereby allocating a unique random number to each publication. Due to the decision to perform a comprehensive search strategy to identify all relevant preclinical animal studies, the majority of the studies were not applicable. To meet the goal of including the 250 relevant publications from each year, publications were imported consecutively in the randomized order (first 500 studies each from the year 2009 and 2018, then 250 studies and lastly 150 studies from each year) into a systematic review manager software program, Covidence (Covidence, Melbourne, Australia) [36]. A total of 1800 studies (out of 3051) were screened for eligibility. Two hundred and fifty-six from 2009 and 275 from 2018 were found eligible. Of these, 250 publications from each year were selected based on the random sampling allocation sequence. The exclusion of studies was based on the following exclusion criteria: science related to farming, wild animals or invertebrates, environment, human (clinical) studies, in vitro research, not primary papers/publications, lack of abstract or full text, studies containing no intervention or no Danish author affiliation, and exploratory studies (the latter studies were identified through study author statements that the study was explorative, or studies were assessed to investigate novel questions and to be hypothesis-generating). Further information on the distribution of excluded studies is found in supporting information (S2 File). The flow diagram for random sampling, screening, and selection of publications is shown in Fig 1.

Data extraction and analysis

The Covidence Risk of Bias (RoB) tool was selectively modified for this study’s aims and to assess reporting quality in compliance with SYRCLE’s RoB tool for animal studies [37]. Each publication was assessed according to 10 items primarily based on the Landis four related to the quality of reporting of significant methodology and included in the ARRIVE guidelines [5, 24, 25]. The selection of items was due to the nature of the study capturing different types of animal research. One item “health status”, was chosen since it, to our knowledge, is scarcely investigated even though it may influence many research outcomes [21]. The reporting quality form and algorithms for scoring are included in the supporting information (S1 Table). Two independent reviewers (KFP and JCS) assessed publications for reporting quality, each blinded to the other’s assessment. Reviewers examined the full text of the articles, including figures and tables, and supplemental information, but references to other studies were not evaluated. Our approach for assessing the reporting quality included three steps:

Step 1: To investigate the overall reporting status (Part I) of the selected items, each item was operationalized such that we scored a result of "Yes" or "No" in Covidence. Publications were qualitatively scored "Yes" if the specific item was reported or "No" when there was no reporting of the item or when criteria for "Yes" were not met. "Unclear" or "Partial" scores were not used. In instances where the item was only partially reported and did not contain the complete information defined in the item or where items were reported not conducted (e.g., authors reported that randomization was not conducted), the study was scored as "Yes" and notes provided. Details of this process are given in the supporting information (S1 Table). Each item’s annotations and quotes were selected and saved for subsequent data quantification. This extra step made judgment decisions during this review consistent.

Step 2: After completing each reviewer’s initial reporting quality assessments, a consensus of the reporting quality results was undertaken in Covidence. If both reviewers agreed on the item, the final judgment defaulted to the agreed value leaving discrepant items for further assessment. Discrepancies were resolved, and the consensus was reached through discussion and the inclusion of a third reviewer (BSK).

Step 3: After completing the assessment in Covidence, data were extracted and sorted in MS Excel. After that, a numerical score of 1, 2, 3, or 0—where 0 corresponds to no information—was given according to the quality of information (quotes and comments) saved for each item described in step 1 (Part II). Details of this process are provided in the supporting information S1 Table, and our criteria for aiding judgment and associated examples of quotes are found in the supporting information (S2 Table).

Survey data were analyzed using MS Excel and Stata Statistical Software: Release 16.1 (Stata Corp. 2019. College Station, TX: StataCorp LLC). Descriptive statistics were generated for all items and were presented in bar graphs and tables. Prevalence and differences between prevalence for the 2009 and 2018 studies were reported with 95% confidence intervals. Reviewer agreement and Cohen’s Kappa values are disclosed in the supporting information (S3 File).

Results

The flow of the publications retrieved is described in the Methods section and in Fig 1. Five hundred publications were included in the investigation, 250 from 2009 and 250 from 2018 according to the procedure described in the Methods section. A simplified two-level scoring (reported “yes” or not reported “no”) is given in part I. Part II discloses the results of a three-level scoring (1, 2, and 3) system where one comprised the least detailed information conveyed. The results of part I are presented graphically in Fig 2 and the results of part II are shown in Table 1.

Fig 2 — *Left Y-axis*: prevalence of reporting in %. *X-axis*: (A) sample size and sample size calculation, (B) randomization (Rand), blinded experiment conduct (Blind EC), and blinded outcome assessment (Blind OA), (C) attrition I (reporting numbers of samples/animals in the result section), attrition II (reporting consistency in numbers of samples/animals between methods and result section), and exclusions (reporting numbers of samples/animals excluded), (D) health status and conflict of interest. The error bars represent 95% confidence intervals. Light grey bars: 2009. Dark grey bars: 2018.

Table 1. Prevalence of reporting details in Danish preclinical research in 2009 compared to 2018.

Item	Details of reporting	2009 (n = 250)		2018 (n = 250)		2018-2009^a
		%	95% CI	%	95% CI	%	95% CI
Sample size calculation	Reported not performed	0.4	[0.0–2.2]	1.2	[0.3–3.5]	0.8	[-0.8–2.4]
	Reported performed	0.4	[0.0–2.2]	5.2	[2.8–8.7]	4.8	[1.9–7.7]
	Reported performed and calculation disclosed	2.4	[0.9–5.2]	7.6	[4.6–11.6]	5.2	[1.4–9.0]
Randomization	Reported performed	22.8	[17.8–28.5]	34.8	[28.9–41.1]	12.0	[4.1–19.9]
Randomization	Reported performed and method disclosed	1.2	[0.3–3.5]	6.0	[3.4–9.7]	4.8	[1.6–8.0]
Blinded experiment conduction	Reported not performed	0.4	[0.0–2.2]	2.8	[1.1–5.7]	2.4	[0.2–4.6]
Blinded experiment conduction	Reported performed	2.0	[0.7–4.6]	1.6	[0.4–4.1]	-0.4	[-2.7–1.9]
Blinded outcome assessment	Reported not performed	0.4	[0.0–2.2]	3.6	[1.7–6.7]	3.2	[0.8–5.6]
Blinded outcome assessment	Reported performed	23.2	[18.1–28.9]	34.4	[28.5–40.7]	11.2	[3.3–19.1]
Attrition I	Reported but not for all analyses	45.6	[39.3–52.0]	52.8	[46.4–59.1]	7.2	[-1.5–15.9]
Attrition I	Reported with exact numbers for all analyses	32.4	[26.6–38.6]	29.6	[24.0–35.7]	-2.8	[-10.9–5.3]
Exclusions	Reported but without numbers and reason for exclusion	6.8	[4.0–10.7]	16.0	[11.7–21.1]	9.2	[3.7–14.7]
	Reported no exclusions/all included	2.0	[0.7–4.6]	4.0	[1.9–7.2]	2.0	[-1.0–5.0]
	Reported with numbers and reason for exclusion	11.6	[7.9–16.2]	18.4	[13.8–23.8]	6.8	[0.6–13.0]
Animal health status	Reported without further information	12.8	[8.9–17.6]	12.4	[8.6–17.1]	-0.4	[-6.2–5.4]
Animal health status	Reported and detailed information disclosed	0.0	[0.0–1.46]	0.4	[0.0–2.2]	0.4	[-0.4–1.2]
Conflict of interest	Reported with a conflict of interest present	6.8	[4.0–10.7]	18.4	[13.8–23.8]	11.6	[5.9–17.3]
Conflict of interest	Reported with a conflict of interest absent	30.8	[25.1–36.9]	72.0	[66.0–77.5]	41.2	[33.2–49.2]

Open in a new tab

CI, confidence interval; n, the total number of publications

^a The relative reporting difference between 2009 and 2018

Part I: The overall reporting status

Approximately half of the reviewed publications from 2009 reported a study sample size given as a number (53.2%, CI 46.8–59.5%) compared to 152 publications (60.8%, CI 54.5–66.9%) in 2018. Eight publications (3.2%, CI 1.4–6.2%) from 2009 reported on sample size calculation as a measure of reassurance that studies were adequately powered. This number increased to 35 publications (14.0%, CI 10.0–18.9%) in 2018 (Fig 2A). Random allocation of animals to experimental groups was reported in 60 publications (24.0%, CI 18.8–29.8%) from 2009 (out of 234 in which this would have been appropriate; 93.6%), which increased to 102 publications (40.8%, CI 34.7–47.2%) in 2018 (out of 233 in which this would have been appropriate; 93.2%). Blinded experiment conduct was reported in six publications (2.4%, CI 0.9–5.2%) from 2009 and increased to 11 in 2018 (4.4%, CI 2.2–7.7%), and blinded outcome assessment was reported in 59 publications (23.6%, CI 18.5–29.4%) from 2009 and increased to 95 (38.0%, CI 32.0–44.3%) in 2018 (Fig 2B).

Information regarding the numbers of samples or animals in the result section (attrition I) was reported in 195 publications (78.0%, CI 72.4–83.0%) from 2009 and increased to 206 (82.4%, CI 77.1–86.9%) in 2018. Consistency in the numbers of samples or animals between the methods and the result section (attrition II) was reported in 18 publications (7.2%, CI 4.3–11.1%) in 2009. This number decreased to 10 publications (4.0%, CI 1.9–7.2%) in 2018.

Exclusions of samples or animals were reported in 51 publications (20.4%, CI 15.6–25.9%) from 2009. In 2018, this number was 96 (38.4%, CI 32.3–44.7%) (Fig 2C). The animal health status was reported in 32 of the publications surveyed (12.8%, CI 8.9–17.6%) in 2009, and the number did not change in 2018 (12.8%, CI 8.9–17.6%). The reporting on conflicts of interest increased from 94 publications (37.6%, CI 31.6–43.9%) in 2009 to 226 publications (90.4%, CI 86.1–93.8%) in 2018 (Fig 2D).

Part II: Level of detail of reported items

Further analysis of information is presented in Table 1 and revealed that of the publications reporting a sample size calculation, six (2.4%) from 2009 provided information regarding how the sample size was chosen and described the method employed. In 2018, this number was 19 (7.6%). The remaining publications that reported on sample size calculation either did not include a calculation method (1 (0.4%) from 2009 and 13 (5.2%) from 2018) or stated that a sample size calculation was not performed (1 (0.4%) from 2009 and 3 (1.2%) from 2018).

Three publications (1.2%) from 2009 reporting random allocation also disclosed the method used. In 2018, the number was 15 publications (6.0%). Notably, 16 (6.4%) and 17 (6.8%) publications from 2009 and 2018, respectively, were either strain studies (e.g., the phenotype of transgenic animals was being compared to wild type phenotype) (2009: 15/16; 2018: 13/17) or pre-post experimental studies (2009: 1/16; 2018: 4/17) where randomizing treatment order was not feasible due to carryover effects. However, none of these publications reported that randomization was not applicable in these types of studies.

Of the publications from 2009 reporting on blinded conduct of the experiment, one publication (0.4%) reported that blinding was not conducted. This increased to seven publications (2.8%) in 2018.

Of the publications reporting on blinded outcome assessment, one publication (0.4%) from 2009 reported that blinding was not conducted. This increased to nine publications (3.6%) in 2018.

Further analysis of information regarding attrition revealed that only 81 (32.4%) of the publications reporting on numbers of samples or animals in the result section (attrition I) from 2009 reported exact numbers for all analyses. This number decreased to 74 (29.6%) in 2018. The remaining publications (114 (45.6%) from 2009 and 132 (52.8%) from 2018) either did not report exact numbers or did not report numbers for all analyses in the study.

For details regarding exclusions of samples or animals, 29 of the publications (11.6%) in 2009 versus 46 (18.4%) in 2018 reported exact numbers and reasons for exclusion. The remaining publications reporting on exclusions either failed to report precise numbers or reasons for exclusion (17 (6.8%) in 2009 and 40 (16.0%) in 2018) or reported no exclusions or all included in the study (five (2.0%) in 2009 and 10 (4.0%) in 2018). Two publications from 2009 and one publication from 2018 provided a flow chart to clarify the animal flow.

When analyzing publications for providing detailed animal health information, one publication (0.4%) from 2018 included a health report of the experimental animals in the supplementary files.

Information regarding conflict of interest showed that 17 publications (6.8%) reported a conflict of interest and 77 publications (30.8%) reported a conflict of interest to be absent in 2009. These numbers increased in 2018 to 46 (18.4%) and 180 (72.0%), respectively.

Discussion

This study investigated the reporting prevalence of central methodological items safeguarding study quality in experimental animal research. The aim was to get an overview of the current status and further research the detail in the information conveyed. To gain further insight into progress over time, we surveyed two time periods, 2009 and 2018. All publications had at least one researcher affiliated with a Danish research institution.

Our results from 2009 correspond well with an investigation from 2010 by Macleod et al., who surveyed preclinical research studies in 2009–2010 from leading UK universities [27]. We also found only modest improvements over time in reporting randomization, blinding, and sample size calculation, whereas the reporting of conflict of interest increased considerably. Our investigation highlights that while this topic has been extensively addressed both in the scientific community and through the development of reporting guidelines [24–31, 38, 39], reporting remains insufficient. There is still considerable room for improvement to strengthen the validity of most published pre-clinical animal studies in the light of the assumption that lack of reporting corresponds to limited conduct.

We further researched the level of detail in the information disclosed and found the level of detail was very limited. Randomization and blinding are essential methodological techniques to help reduce the influence of bias on the study outcome. Despite their importance, transparency in reporting these items was insufficient. In studies where blinding and randomization were not feasible, the reason (e.g., study design) was rarely justified nor considered a limitation and acknowledged in the study report. A description of why such a precaution is not taken will bring the reader’s attention to the missing safeguard so the results can be judged accordingly. Many studies additionally have very complex study designs and the precautions taken to limit bias should be sufficiently reported.

In general, essential details related to randomization, such as the allocation method and sequence, were rarely conveyed. This fact is corroborated in studies of specific animal models of acute lung injury by Avey et al. They operationalized the ARRIVE guidelines to determine completeness in reporting and found no random sequence generation reporting [30]. Ting et al. similarly disclosed that no studies revealed the allocation method in experimental animal studies of rheumatology [28]. Our investigation concludes that there seems to be a general challenge across study fields.

Interestingly, small sample sizes may negatively influence successful randomization as groups may be unbalanced on critical prognostic variables. Underpowered experiments will give less precise estimates of treatment effects. This risk can be accounted for by using appropriate methods for sample size calculation.

Only a few publications provided sufficient information regarding if and how sample size was calculated for sample sizes’ exact values. In some publications, historical precedent rather than reliable statistics formed the basis for reporting the number of animals per group. We are puzzled that such unjustified scientific information is forwarded through a review process. In a study by Gulin et al., investigating compliance with ARRIVE guidelines in studies of experimental animal models for Chagas disease, there was no reporting of sample size calculation. Authors of the investigation speculated that "animal numbers were more a matter of habit than a statistical decision" [29]. This speculation highlights that results may sometimes be due to chance rather than an actual effect. A more recent study of the veterinary literature that focused on reporting adherence to the ARRIVE guidelines found missing sample size calculations to be present in both ARRIVE guideline supporting and non-supporting journals, indicating that a journal’s support for ARRIVE guidelines has not to date resulted in improved reporting of these guidelines and other essential indicators of study design quality [31]. If the planned sample size is not derived statistically, this should be explicitly stated along with the rationale for the intended sample size (e.g., exploratory nature). We found information on whether a study was confirmatory or exploratory sparse. This information poses an additional problem to how much weight can be ascribed to the published results.

Systematic differences between animals completing a study and the excluded animals can introduce bias to the study results–a bias known as attrition bias [17]. Despite the importance of emphasizing and reporting exact numbers of animals at the beginning of the study and the end of the study and how many animals were excluded during the study and for which reasons, we found most studies failed to report consistently. Most publications failed to report exact numbers and reasons for exclusion, and even a decrease in reporting of animal numbers in 2018 was seen when compared with 2009. Several studies reporting on the number of samples or animals used demonstrated inconsistencies in reporting between the methods section and the results section. Only one publication from 2018 included this information objectively in a flow chart compared to two publications in 2009. A flowchart illustrating each animal’s fate and the derived samples or measurements would be effective in providing the reader with a thorough overview.

An uncommonly reported item and, to our knowledge, rarely investigated item is the animals’ general health status. In our study, this was one of the most poorly reported items, and only one publication from 2018 included a health report with details of the specific agents for which the animals were screened. This finding is disturbing since infections and/or comorbidities influence disease outcomes in both preclinical animal research and treatment and pathology in patients [40]. Documenting these details is essential in understanding the discrepancies seen in laboratory results [25, 41]. In our experience, many researchers do not take this fact into account. A fully disclosed health report should be mandatory and based on a case-oriented approach to the FELASA (Federation of European Laboratory Animal Science Associations) guidelines [22, 23]. Moreover, the impact of animal health on study outcomes is complex and warrants further investigation.

We envisaged that an improvement in methodological reporting would be noticeable since many journals have endorsed the ARRIVE guidelines. However, advancements continue to progress at a slow pace or do not happen at all. We show that the reported information’s level of detail is generally incomplete. The incomplete reporting of these details directly impedes the ability to assess the validity of the experiments. When research cannot be assessed on its methodological rigor, it becomes less valuable and thus is a waste of essential resources and animal lives. The translation of research findings into therapeutic applications becomes highly unreliable, and there is a high risk of guiding research in the wrong direction. Stakeholders such as funders and publishers may incite study quality, but perhaps the essential science stakeholders are researchers themselves. Researchers must conduct responsible research of high quality and have the ability to do so. This may also call for new evaluation methods [42].

Nevertheless, to conduct high-quality research, researchers need to be allocated time, understand the importance of research integrity, be trained in best practices, and know about the available tools, such as guidelines for planning and conducting animal-based studies [25, 43]. Recently, a case study demonstrated the impact of conducting preclinical systematic reviews on the quality and transparency of research and researchers’ awareness and motivation to promote change within their fields [44]. A critical comment was that many had not previously known how to report their research adequately, nor had they realized the importance of accurate reporting. Through systematic reviews, they became aware of the low reporting quality, and they became completer and more precise in the way they planned, executed, and reported their study. They also changed their view on the necessity to improve their team and research field. Hence, the assumption that this topic is well known and recognized among researchers may be wrong. There seems to be a need for more thorough education within this science field to implement rigor in one’s preclinical animal study.

To accelerate progress, we conclude that educational institutions must look closer to home and support and increase educational activities of relevant teaching and training in designing and reporting animal studies. It is disappointing that large prestigious public research institutions fail to adequately report study characteristics (and also that the institutions assess publications with poor study quality at the same level as publications with high study quality). Proper education is necessary, and knowledge from education in systematic review methodology and conduction of randomized clinical control studies may guide how to approach the topic. Initiatives such as collaborative research groups and networks that serve as a backbone for this strategy should be prioritized.

Supporting information

S1 File. Protocol.

(DOCX)

Click here for additional data file.^{(43KB, docx)}

S2 File. Supplementary material.

(DOCX)

Click here for additional data file.^{(16.9KB, docx)}

S3 File. Agreement data.

(XLSX)

Click here for additional data file.^{(18.3KB, xlsx)}

S4 File. PRISMA 2020 checklist.

(DOCX)

Click here for additional data file.^{(32.4KB, docx)}

S1 Table. Data extraction form.

(DOCX)

Click here for additional data file.^{(15.2KB, docx)}

S2 Table. Criteria for the assessment of reporting quality.

(DOCX)

Click here for additional data file.^{(21KB, docx)}

Acknowledgments

We are grateful to Professor Malcolm Macleod, Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, the United Kingdom for inspiring to the project, to Morten Frydenberg, Department of Public Health at Aarhus University in Denmark, for advice on random sampling, statistical analyses and expert advice on the statistical graphs, and to Karen Rodríguez Sigaard, Aarhus University Library in Denmark, for aiding in the setup and modification of search strings. Dr. Alexandra Bannach-Brown from the Collaborative Approach to Meta Analyses and Review of Animal Experimental studies (CAMARADES Research Group) is acknowledged for her contributions during her time at Aarhus University.

Data Availability

All data files are available from the Open Science Framework database: https://osf.io/7e829/.

Funding Statement

BSK received funding from the Danish 3R Center, grant number 33010-NIFA-19-705, https://en.3rcenter.dk/ and Ester M. og Konrad Kristian Siggurdssons Dyrevaernsfond, https://www.dyrevaernsfond.com/ The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

References

1.Baker M., 1,500 scientists lift the lid on reproducibility. Nature, 2016. 533(7604): p. 452–4. doi: 10.1038/533452a [DOI] [PubMed] [Google Scholar]
2.Begley C.G. and Ellis L.M., Raise standards for preclinical cancer research. Nature, 2012. 483(7391): p. 531–533. [DOI] [PubMed] [Google Scholar]
3.Voelkl B., et al., Reproducibility of animal research in light of biological variation. Nature Reviews Neuroscience, 2020. 21(7): p. 384–393. doi: 10.1038/s41583-020-0313-3 [DOI] [PubMed] [Google Scholar]
4.van der Worp H.B., et al., Can animal models of disease reliably inform human studies? PLoS medicine, 2010. 7(3): p. e1000245–e1000245. doi: 10.1371/journal.pmed.1000245 [DOI] [PMC free article] [PubMed] [Google Scholar]
5.Landis S.C., et al., A call for transparent reporting to optimize the predictive value of preclinical research. Nature, 2012. 490(7419): p. 187–91. doi: 10.1038/nature11556 [DOI] [PMC free article] [PubMed] [Google Scholar]
6.Glasziou P., et al., Reducing waste from incomplete or unusable reports of biomedical research. The Lancet, 2014. 383(9913): p. 267–276. doi: 10.1016/S0140-6736(13)62228-X [DOI] [PubMed] [Google Scholar]
7.Tikka C., et al., Quality of reporting and risk of bias: a review of randomised trials in occupational health. Occupational and Environmental Medicine, 2021. 78(9): p. 691–696. doi: 10.1136/oemed-2020-107038 [DOI] [PMC free article] [PubMed] [Google Scholar]
8.Riley S.P., et al., A systematic review of orthopaedic manual therapy randomized clinical trials quality. The Journal of manual & manipulative therapy, 2016. 24(5): p. 241–252. doi: 10.1080/10669817.2015.1119372 [DOI] [PMC free article] [PubMed] [Google Scholar]
9.Jerndal M., et al., A systematic review and meta-analysis of erythropoietin in experimental stroke. J Cereb Blood Flow Metab, 2010. 30(5): p. 961–8. doi: 10.1038/jcbfm.2009.267 [DOI] [PMC free article] [PubMed] [Google Scholar]
10.Vesterinen H.M., et al., Improving the translational hit of experimental treatments in multiple sclerosis. Multiple Sclerosis Journal, 2010. 16(9): p. 1044–1055. doi: 10.1177/1352458510379612 [DOI] [PubMed] [Google Scholar]
11.Macleod M.R., et al., Evidence for the Efficacy of NXY-059 in Experimental Focal Cerebral Ischaemia Is Confounded by Study Quality. Stroke, 2008. 39(10): p. 2824–2829. doi: 10.1161/STROKEAHA.108.515957 [DOI] [PubMed] [Google Scholar]
12.Hirst J.A., et al., The need for randomization in animal trials: an overview of systematic reviews. PLoS One, 2014. 9(6): p. e98856. doi: 10.1371/journal.pone.0098856 [DOI] [PMC free article] [PubMed] [Google Scholar]
13.Chalmers T.C., et al., Bias in treatment assignment in controlled clinical trials. N Engl J Med, 1983. 309(22): p. 1358–61. doi: 10.1056/NEJM198312013092204 [DOI] [PubMed] [Google Scholar]
14.Schulz K.F., et al., Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. Jama, 1995. 273(5): p. 408–12. doi: 10.1001/jama.273.5.408 [DOI] [PubMed] [Google Scholar]
15.Wood L., et al., Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study. Bmj, 2008. 336(7644): p. 601–5. doi: 10.1136/bmj.39465.451748.AD [DOI] [PMC free article] [PubMed] [Google Scholar]
16.Hróbjartsson A., et al., Observer bias in randomized clinical trials with measurement scale outcomes: a systematic review of trials with both blinded and nonblinded assessors. Cmaj, 2013. 185(4): p. E201–11. doi: 10.1503/cmaj.120744 [DOI] [PMC free article] [PubMed] [Google Scholar]
17.Holman C., et al., Where Have All the Rodents Gone? The Effects of Attrition in Experimental Research on Cancer and Stroke. PLoS biology, 2016. 14(1): p. e1002331–e1002331. doi: 10.1371/journal.pbio.1002331 [DOI] [PMC free article] [PubMed] [Google Scholar]
18.Button K.S., et al., Power failure: why small sample size undermines the reliability of neuroscience. Nat Rev Neurosci, 2013. 14(5): p. 365–76. doi: 10.1038/nrn3475 [DOI] [PubMed] [Google Scholar]
19.Cressey D., UK funders demand strong statistics for animal studies. Nature, 2015. 520(7547): p. 271–2. doi: 10.1038/520271a [DOI] [PubMed] [Google Scholar]
20.Ioannidis J.P.A., Why Most Published Research Findings Are False. PLOS Medicine, 2005. 2(8): p. e124. doi: 10.1371/journal.pmed.0020124 [DOI] [PMC free article] [PubMed] [Google Scholar]
21.Baker D.G., Natural pathogens of laboratory mice, rats, and rabbits and their effects on research. Clinical microbiology reviews, 1998. 11(2): p. 231–266. doi: 10.1128/CMR.11.2.231 [DOI] [PMC free article] [PubMed] [Google Scholar]
22.Mähler Convenor M., et al., FELASA recommendations for the health monitoring of mouse, rat, hamster, guinea pig and rabbit colonies in breeding and experimental units. Lab Anim, 2014. 48(3): p. 178–192. doi: 10.1177/0023677213516312 [DOI] [PubMed] [Google Scholar]
23.Berset Convenor F., et al., Federation of European Laboratory Animal Science Associations recommendations of best practices for the health management of ruminants and pigs used for scientific and educational purposes. Lab Anim, 2021. 55(2): p. 117–128. doi: 10.1177/0023677220944461 [DOI] [PMC free article] [PubMed] [Google Scholar]
24.Kilkenny C., et al., Improving bioscience research reporting: the ARRIVE guidelines for reporting animal research. PLoS Biol, 2010. 8(6): p. e1000412. doi: 10.1371/journal.pbio.1000412 [DOI] [PMC free article] [PubMed] [Google Scholar]
25.Percie du Sert N., et al., The ARRIVE guidelines 2.0: Updated guidelines for reporting animal research. BMC Veterinary Research, 2020. 16(1): p. 242. doi: 10.1186/s12917-020-02451-y [DOI] [PMC free article] [PubMed] [Google Scholar]
26.Baker D., et al., Two Years Later: Journals Are Not Yet Enforcing the ARRIVE Guidelines on Reporting Standards for Pre-Clinical Animal Studies. PLOS Biology, 2014. 12(1): p. e1001756. doi: 10.1371/journal.pbio.1001756 [DOI] [PMC free article] [PubMed] [Google Scholar]
27.Macleod M.R., et al., Risk of Bias in Reports of In Vivo Research: A Focus for Improvement. PLoS Biol, 2015. 13(10): p. e1002273. doi: 10.1371/journal.pbio.1002273 [DOI] [PMC free article] [PubMed] [Google Scholar]
28.Ting K.H.J., Hill C.L., and Whittle S.L., Quality of reporting of interventional animal studies in rheumatology: a systematic review using the ARRIVE guidelines. International Journal of Rheumatic Diseases, 2015. 18(5): p. 488–494. doi: 10.1111/1756-185X.12699 [DOI] [PubMed] [Google Scholar]
29.Gulin J.E.N., Rocco D.M., and García-Bournissen F., Quality of Reporting and Adherence to ARRIVE Guidelines in Animal Studies for Chagas Disease Preclinical Drug Research: A Systematic Review. PLOS Neglected Tropical Diseases, 2015. 9(11): p. e0004194. doi: 10.1371/journal.pntd.0004194 [DOI] [PMC free article] [PubMed] [Google Scholar]
30.Avey M.T., et al., The Devil Is in the Details: Incomplete Reporting in Preclinical Animal Research. PLoS One, 2016. 11(11): p. e0166733. doi: 10.1371/journal.pone.0166733 [DOI] [PMC free article] [PubMed] [Google Scholar]
31.Leung V., et al., ARRIVE has not ARRIVEd: Support for the ARRIVE (Animal Research: Reporting of in vivo Experiments) guidelines does not improve the reporting quality of papers in animal welfare, analgesia or anesthesia. PLOS ONE, 2018. 13(5): p. e0197882. doi: 10.1371/journal.pone.0197882 [DOI] [PMC free article] [PubMed] [Google Scholar]
32.Macleod M.R., et al., Correction: Risk of Bias in Reports of In Vivo Research: A Focus for Improvement. PLoS Biol, 2015. 13(11): p. e1002301. doi: 10.1371/journal.pbio.1002301 [DOI] [PMC free article] [PubMed] [Google Scholar]
33.Vries R.B.M., et al., A protocol format for the preparation, registration and publication of systematic reviews of animal intervention studies. Evidence-based Preclinical Medicine, 2015. 2(1): p. e00007. [Google Scholar]
34.Leenaars M., et al., A step-by-step guide to systematically identify all relevant animal studies. Lab Anim, 2012. 46(1): p. 24–31. doi: 10.1258/la.2011.011087 [DOI] [PMC free article] [PubMed] [Google Scholar]
35.Hooijmans C.R., et al., Enhancing search efficiency by means of a search filter for finding all studies on animal experimentation in PubMed. Laboratory Animals, 2010. 44(3): p. 170–175. doi: 10.1258/la.2010.009117 [DOI] [PMC free article] [PubMed] [Google Scholar]
36.Covidence systematic review software. [cited 2020/ 11/12]; www.covidence.org.
37.Hooijmans C.R., et al., SYRCLE’s risk of bias tool for animal studies. BMC Med Res Methodol, 2014. 14: p. 43. doi: 10.1186/1471-2288-14-43 [DOI] [PMC free article] [PubMed] [Google Scholar]
38.Han S., et al., A checklist is associated with increased quality of reporting preclinical biomedical research: A systematic review. PLOS ONE, 2017. 12(9): p. e0183591. doi: 10.1371/journal.pone.0183591 [DOI] [PMC free article] [PubMed] [Google Scholar]
39.Hair K., et al., A randomised controlled trial of an Intervention to Improve Compliance with the ARRIVE guidelines (IICARus). Research Integrity and Peer Review, 2019. 4(1): p. 12. [DOI] [PMC free article] [PubMed] [Google Scholar]
40.Smith C.J., et al., The immune system in stroke: clinical challenges and their translation to experimental research. Journal of neuroimmune pharmacology: the official journal of the Society on NeuroImmune Pharmacology, 2013. 8(4): p. 867–887. doi: 10.1007/s11481-013-9469-1 [DOI] [PubMed] [Google Scholar]
41.Percie du Sert N., et al., Reporting animal research: Explanation and elaboration for the ARRIVE guidelines 2.0. PLOS Biology, 2020. 18(7): p. e3000411. doi: 10.1371/journal.pbio.3000411 [DOI] [PMC free article] [PubMed] [Google Scholar]
42.Ioannidis J.P. and Khoury M.J., Assessing value in biomedical research: the PQRST of appraisal and reward. Jama, 2014. 312(5): p. 483–4. doi: 10.1001/jama.2014.6932 [DOI] [PMC free article] [PubMed] [Google Scholar]
43.Smith A.J., et al., PREPARE: guidelines for planning animal research and testing. Lab Anim, 2018. 52(2): p. 135–141. doi: 10.1177/0023677217724823 [DOI] [PMC free article] [PubMed] [Google Scholar]
44.Menon J.M.L., et al., The impact of conducting preclinical systematic reviews on researchers and their research: A mixed method case study. PloS one, 2021. 16(12): p. e0260619–e0260619. doi: 10.1371/journal.pone.0260619 [DOI] [PMC free article] [PubMed] [Google Scholar]

PLoS One. doi: 10.1371/journal.pone.0275962.r001

Decision Letter 0

Gillian Currie

10 Mar 2022

PONE-D-21-20844Reporting quality in preclinical animal experimental research in 2009 and 2018: A nationwide systematic investigation

PLOS ONE

Dear Dr. Kousholt,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

I have had difficulties securing peer reviewers for your study therefore I have made an exception that my decision will be based on my own review and one other independent peer review.

As well as addressing my specific comments and those from the independent peer reviewer, I request that the raw data for individual publications is made fully available without restriction either as part of the manuscript or in a public repository.

I suggest that the discussion and conclusions should reflect that these findings are specific to Danish researchers and how this compares to the findings of other similar studies sampling from other countries or without institutional restrictions. In other words, emphasize the differences and similarities between your study and previous research.

Here are some specific sentences that require clarification or edits.

Abstract

Line 20-21: “Publications from two periods with at least one affiliation to a 21 Danish university conveys any reporting progress.” Amend sentence to clarify meaning.

Line 40-42: “Knowledge on why adequate planning, execution and reporting are of importance. We suggest thorough 42 teaching in designing and reporting animal studies.” Amend sentences to clarify meaning.

Introduction

Line 49-52: “A prevalent hindrance in reproducing experiments and lack of translation to bedside is an unsatisfactory internal validity. It refers to the extent to which appropriate methodologies safeguarding against systematic errors (bias) are implemented in the design, conduct, and analysis of an experiment” Amend sentences to clarify meaning.

Line 74-75: “In this context, realization is necessary already at the planning stages to guarantee good reporting in the end.” Amend sentence to clarify meaning.

Selection of studies

Line 122: Please give details on how you defined and identified an exploratory study.

Data extraction and analysis

Line 164: “Differences between prevalence for the 2009 and 2018 studies were 164 reported with approximate 95% confidence intervals.” Do we need the word approximate? Or are you reporting the 95% confidence intervals?

Line 337: “We envisaged that an improvement in methodological reporting would be noticeable since so many journals adopt the ARRIVE guidelines.” I suggest that this sentence should be edited to read “We envisaged that an improvement in methodological reporting would be noticeable since many journals have endorsed the ARRIVE guidelines.”

Line 370: “Professor Malcolm MacLeod” Should read Professor Malcolm Macleod.

Please submit your revised manuscript by Apr 24 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Gillian Currie

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. We note that this manuscript is a systematic review or meta-analysis; our author guidelines therefore require that you use PRISMA guidance to help improve reporting quality of this type of study. Please upload copies of the completed PRISMA checklist as Supporting Information with a file name “PRISMA checklist”.

3. Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

Additional Editor Comments (if provided):

Here are some specific sentences that require clarification or edits.

Abstract

Line 20-21: “Publications from two periods with at least one affiliation to a 21 Danish university conveys any reporting progress.” Amend sentence to clarify meaning.

Introduction

Line 74-75: “In this context, realization is necessary already at the planning stages to guarantee good reporting in the end.” Amend sentence to clarify meaning.

Selection of studies

Line 122: Please give details on how you defined and identified an exploratory study.

Data extraction and analysis

Line 370: “Professor Malcolm MacLeod” Should read Professor Malcolm Macleod.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: This is an interesting study evaluating the reporting quality of preclinical studies from academic institutions in Denmark. The authors assess quality at a timepoint prior to the ARRIVE guidelines and a more recent timepoint in 2018. Overall, this is a worthwhile effort to determine if there have been improvements in reporting quality over time, focussing on measures to reduce the risk of systematic bias in animal experiments. Given the findings of other studies, the results are not surprising and indicate that reporting of key information relating to the internal validity of a study is often omitted. In fact, the findings suggest that the reporting of many quality items has not improved substantially since 2009.

General comments:

There is a need to emphasise throughout the aims and perhaps also in the discussion how this study differs from previous studies. I think one of the key strengths is that you looked in more detail at the information provided e.g. whether the randomisation method was described.

Data availability - would you be wiling to share the raw publication level data with the scores for each paper? If extracted data were available for each paper it could (1) enable future efforts to assess reporting and (2) act as training/test data for data scientists who are creating tools to automatically assess papers for risk of bias reporting.

Introduction

Line 49: Perhaps amend this sentence as it does not read well. As reproducibility and translation have just been mentioned, you could shorten to “A prevalent issue is unsatisfactory internal validity”

Line 53: Attrition is described as a safeguard here which is confusing – is there another way to phrase this?

Line 54-5: “Evidence exists that lack of reporting corresponds to the absence of conduct.” – could you add a citation to support this?

Line 57: “compared with studies not taking these precautions” – change to not reporting these precautions as we cannot be sure that researchers did not control for biases

Line 57-58: I am unclear what this means and unsure how relevant this is to the rest of the narrative

As you mention, there have been many previous studies looking at reporting quality in preclinical research. Therefore, it would be good to set out how your study differs from existing research in greater detail. Your study is focussed on specific Danish research institutions so it would be useful to explain the rationale behind this and refer to the landscape of preclinical research within Denmark.

Methods

Line 86: Was this estimation based on previous research or calculated in some way?

Line 98: I see that the search strategy is based on the SYRCLE animal filter, but could it also be shown in full in the supplementary information?

Line 101: What are the 5 Danish universities and why were they selected?

Line 105: Inconsistencies in study number descriptions (written in text for 2009, then numerical values for 2018).

These values would be better placed in the results section for clarity

For the quality of reporting items, why did you select these 10? As you mention several risk of bias tools and the ARRIVE guidelines, it is unclear how you arrived at this list and didn’t include the sex of the animals for example (from ARRIVE essential 10) but included health status. I suspect this may be due to the nature of the study capturing all different types of animal research, but it would be good to add some further information on this decision.

Results

Line 169: Repetition of prevalence being reported with 95% confidence intervals

Line 155: Was this score (0-3) used for any summary statistics or analysis?

Line 182: for random allocation, the percentage values have been calculated based on the full 250 papers rather than the subset of applicable papers which seems incorrect

I was going to suggest kappa values for reviewer agreement and see that an analysis has been performed as shown in the supplementary data file – it may be good to include a summary of this in the manuscript and highlight items which reviewers did not always agree on.

Discussion:

The discussion is well written and raises some good points. I wonder whether there could again be a mention of Denmark here and how these findings fit in to any country-specific research improvement activities and how the findings may also apply to other European institutions.

You mention here that the 5 institutions investigated are large public research institutions – I wondered whether it would be feasible to do a post-hoc analysis of study quality between these institutions, which may provide some insight into what a specific university may be doing right vs wrong.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PLoS One. 2022 Nov 3;17(11):e0275962. doi: 10.1371/journal.pone.0275962.r002

Author response to Decision Letter 0

26 May 2022

Response to reviewers:

Dear Academic Editor, Dr. Gilllian Currie and Reviewer #1

We appreciate the efforts put into reviewing our paper (PONE-D-21-20844) and thank you for your comments. The paper has been revised accordingly. Please find our point-by-point reply below. In addition to this rebuttal letter labeled 'Response to Reviewers', we have uploaded a marked-up copy of the manuscript that highlights changes made to the original version labeled 'Revised Manuscript with Track Changes', and an unmarked version of the revised paper without tracked changes labeled 'Manuscript' as requested.

Shortly after the upload we received an email concerning a request for a PRISMA check list. We have uploaded a checklist but do however feel that this may be a bit wrong to do so. Please find our reply to this request below in response to journal requirements item 2.

Response to Dr Currie’s comments and requests:

Response:

We agree that access to raw data is of value to the scientific community. We would however, like to work further with the data ourselves and wonder if PLOS One has any experience with delayed release of raw data? We are currently looking into the best way to make data accessible.

You present an interesting study assessing the reporting of measures to reduce the risk of bias in 2009 and 2018 in a random sample of studies with at least one author affiliated with a Danish institution. I suggest that the discussion and conclusions should reflect that these findings are specific to Danish researchers and how this compares to the findings of other similar studies sampling from other countries or without institutional restrictions. In other words, emphasize the differences and similarities between your study and previous research.

Response:

Thank you very much for this comment. We agree that it is important to let the discussion and conclusion reflect the study’s specific findings compared to other similar studies. We have emphasized similarities and differences in the following:

Line 72-78: “Previous research has investigated the prevalence of reporting of measures to reduce the risk of bias for specific animal disease models or subjects of interest [29-32]. Other previous evaluations of preclinical reporting have provided an overview of the reporting status of items related to the internal validity or rigor of these experiments (e.g. blinding and randomization) [33]. Taking a step further, this study investigates the reported information's level of detail by assessing preclinical studies within all animal experimental research fields with one or more authors affiliated with Danish research institutions.”.

Line 281-282: “We further researched the level of detail in the information disclosed and found the level of detail was very limited.”

Line 288-289: “Many studies additionally have very complex study designs and the precautions taken to limit bias should be sufficiently reported.”

Line 291-293: “In general, essential details related to randomization, such as the allocation method and sequence, were rarely conveyed. This fact is corroborated in studies of specific animal models of acute lung injury by Avey et al. They operationalized the ARRIVE guidelines to determine completeness in reporting and found no random sequence generation reporting [31]. Ting et al. similarly disclosed that no studies revealed the allocation method in experimental animal studies of rheumatology [29]. Our investigation concludes that there seems to be a general challenge across study fields.

Line 312-313: “A more recent study of the veterinary literature that focused on reporting adherence to the ARRIVE guidelines…”

Line 349-351: “We show that the reported information’s level of detail is generally incomplete. The incomplete reporting of these details directly impedes the ability to assess the validity of the experiments.”.

Response to specific sentences that require clarification or edits.

Abstract

Line 20-21: “Publications from two periods with at least one affiliation to a 21 Danish university conveys any reporting progress.” Amend sentence to clarify meaning.

Response:

To clarify meaning we have amended the sentence in line 19-21 to “Publications were from two time periods to convey any reporting progress and had at least one author affiliated to a Danish University.”.

Response:

To clarify meaning we have edited the sentence to: “We suggest thorough teaching in designing, conducting and reporting animal studies. Education in systematic review methodology should be implemented in this training and will increase motivation and behavior working towards quality improvements in science.”. in line 35-38.

Introduction

Response:

Thank you for noticing this. Instead of “it refers..” the sentence now includes “the internal validity”. According to reviewer one’s request, “a prevalent hindrance in reproducing experiments and lack of translation to bedside” has been changed to “..issue..”. The sentences are edited to “A prevalent issue is unsatisfactory internal validity [4]. Internal validity refers to how appropriate methodologies safeguarding against systematic errors (bias) are implemented in the design, conduct, and analysis of an experiment [5]. in line 43-45.

Line 74-75: “In this context, realization is necessary already at the planning stages to guarantee good reporting in the end.” Amend sentence to clarify meaning.

Response:

Thank you for this comment. We have clarified the meaning by rewriting the sentence:

Line 68-71: “The implementation may be hindered by the lack of engagement of multiple stakeholders who all must engage in improving the reporting quality. In this context, researchers’ use of the guideline is necessary already in the planning stages to guarantee good reporting.”.

Materials and methods

Selection of studies

Line 122: Please give details on how you defined and identified an exploratory study.

Response:

We defined an exploratory study as research connecting ideas to understand cause-effect and investigating novel relevant questions that have not previously been thoroughly studied. These studies do not test but generate hypotheses.

To clarify this we have added “…(i.e. studies investigating novel questions and hypothesis-generating studies)…”. The sentence now states:

Line 119-123: “The exclusion of studies was based on the following exclusion criteria: science related to farming, wild animals or invertebrates, environment, human (clinical) studies, in vitro research, not primary papers/publications, lack of abstract or full text, exploratory studies (i.e. studies investigating novel questions and hypothesis-generating studies), and studies containing no intervention or no Danish author affiliation.”

Data extraction and analysis

Response:

Thank you for pointing this out. The word approximate is not necessary and have been omitted.

Line 167-168: “Prevalence and differences between prevalence for the 2009 and 2018 studies were reported with 95% confidence intervals.”

Response:

Thank you. The sentence is corrected according to your request.

Line 347-348: “We envisaged that an improvement in methodological reporting would be noticeable since many journals have endorsed the ARRIVE guidelines.”

Line 370: “Professor Malcolm MacLeod” Should read Professor Malcolm Macleod.

Response:

Thank you for pointing out this misspelling. This is now corrected to read “Professor Malcolm Macleod” in line 382.

Response to journal requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

Response:

We have revisited PLOS ONE’s style requirements as well as followed the templates for correcting the formatting:

Line 14: “BSK” is added to the corresponding author email

Line 16, 22, 29, 37: We have deleted the subtitles “objective”, “methods”, “results, and “conclusion” from the abstract.

Line 128-129: Figure title is written in bold type

Line 179: “table 1” is corrected to “Table 1”

Line 262: Table 1 title is written in bold type, 12 pt

Line 263-264: table and legend are written in 12 pt

In addition, supporting information files have been renamed to match the supporting information captions within the manuscript.

Response:

Thank you for this comment. It leads to some confusion. We do not perceive the manuscript as a systematic review per se and several items from the PRISMA checklist is not applicable. This is due to the fact that no summary of all individual studies over a specific health-related issue is given. Furthermore, the study does not inform decisions about the safety and efficacy of a treatment for participants in clinical trials but rather convey the reporting quality and progress in reporting in preclinical studies. We do however agree that part of the systematic review methodology is applied and this leads to the results presented in this investigation.

Early on in the submission process we were in contact with PLOS ONE and received a reply in august 2021 confirming that this view was accepted. If you have changed your view on this please let us now for further discussion. We have for the sake of moving the manuscript further uploaded the checklist with file name “S4_file.word”. In the manuscript we have added “S4 File. PRISMA checklist” in line 500.

Response:

The following two references have been added to the reference list (reference number 8 and 9 in line 52 in the manuscript)

Line 406-407: Tikka, C., et al., Quality of reporting and risk of bias: a review of randomised trials in occupational health. Occupational and Environmental Medicine, 2021. 78(9): p. 691-696.

Line 408-409 :Riley, S.P., et al., A systematic review of orthopaedic manual therapy randomized clinical trials quality. The Journal of manual & manipulative therapy, 2016. 24(5): p. 241-252

The following two references have been edited:

Line 481-482: The reference “Hair, K., M.R. Macleod, and E.S. Sena, A randomised controlled trial of an Intervention to Improve Compliance with the ARRIVE guidelines (IICARus). bioRxiv, 2018” is now published in Research Integrity and Peer Review doi: 10.1186/s41073-019-0069-3. Thus the reference has been updated to “Hair, K., et al., A randomised controlled trial of an Intervention to Improve Compliance with the ARRIVE guidelines (IICARus). Research Integrity and Peer Review, 2019. 4(1): p. 12.”

Line 492-494: The reference number 45 “Z.s. Modules - Programme "More knowledge with fewer animals". 2020; Available from: https://www.zonmw.nl/nl/actueel/nieuws/detail/item/systematisch-literatuuronderzoek-vervangt-vermindert-en-verfijnt-proefdieronderzoek/.” is replaced since data from this case study has been published in 2021. The reference number 45 in line 363 in the manuscript is corrected to: Menon, J.M.L., et al., The impact of conducting preclinical systematic reviews on researchers and their research: A mixed method case study. PloS one, 2021. 16(12): p. e0260619-e0260619.

Reviewers' comments:

Reviewer #1: This is an interesting study evaluating the reporting quality of preclinical studies from academic institutions in Denmark. The authors assess quality at a timepoint prior to the ARRIVE guidelines and a more recent time point in 2018. Overall, this is a worthwhile effort to determine if there have been improvements in reporting quality over time, focusing on measures to reduce the risk of systematic bias in animal experiments. Given the findings of other studies, the results are not surprising and indicate that reporting of key information relating to the internal validity of a study is often omitted. In fact, the findings suggest that the reporting of many quality items has not improved substantially since 2009.

Response to general comments.

Response:

Thank you for this comment. We have responded to this matter earlier on in this letter. Please refer to the earlier response beginning with: “Thank you very much for this comment. We agree that it is important to let the discussion and conclusion reflect the study’s specific findings compared to other similar studies. We have emphasized similarities and differences in the following…” p. 2 in this letter.

Data availability - would you be willing to share the raw publication level data with the scores for each paper? If extracted data were available for each paper it could (1) enable future efforts to assess reporting and (2) act as training/test data for data scientists who are creating tools to automatically assess papers for risk of bias reporting.

Response:

Introduction

Response:

The sentence is edited to “A prevalent issue is unsatisfactory internal validity” in line 43.

Line 53: Attrition is described as a safeguard here which is confusing – is there another way to phrase this?

Response:

Attrition is now rephrased to “description of animals’ and samples’ flow including reasons for exclusion

Line 46-47 is edited to “Essential safeguards are blinding, randomization, and a thorough description of animals’ and samples’ flow including reasons for exclusion [6].”.

Line 54-5: “Evidence exists that lack of reporting corresponds to the absence of conduct.” – could you add a citation to support this?

Response:

The main issue with poor reporting is that risk of bias becomes difficult to assess. So this usually leads to unclear (or some concerns) risk of bias judgments for the overall study. We have added two references from clinical trials “[8, 9]” in line 49 showing that poorly reported trials had a higher risk of bias.

Line 57: “compared with studies not taking these precautions” – change to not reporting these precautions as we cannot be sure that researchers did not control for biases

Response:

The word “taking” has been corrected to “reporting” in line 51.

Line 57-58: I am unclear what this means and unsure how relevant this is to the rest of the narrative.

As you mention, there have been many previous studies looking at reporting quality in preclinical research. Therefore, it would be good to set out how your study differs from existing research in greater detail. Your study is focused on specific Danish research institutions so it would be useful to explain the rationale behind this and refer to the landscape of preclinical research within Denmark.

Response:

Thank you for these comments. Regarding the meta-epidemiological studies from clinical data, we regard them as important since they corroborate findings in preclinical research and furthermore further researches the additive effect of more than one bias. We anticipate that this is equivalent in preclinical research.

Line 51-53: “This finding is corroborated in meta-epidemiological studies of clinical data that identify a negative additive impact when more than one safeguard is omitted [14-17].”.

In terms of how our study differs from existing research we have emphasized this in the following:

Line 281-282: “We further researched the level of detail in the information disclosed and found the level of detail was very limited.”

Line 288-289: “Many studies additionally have very complex study designs and the precautions taken to limit bias should be sufficiently reported.”

Line 291-297: “In general, essential details related to randomization, such as the allocation method and sequence, were rarely conveyed. This fact is corroborated in studies of specific animal models of acute lung injury by Avey et al. They operationalized the ARRIVE guidelines to determine completeness in reporting and found no random sequence generation reporting [31]. Ting et al. similarly disclosed that no studies revealed the allocation method in experimental animal studies of rheumatology [29]. Our investigation concludes that there seems to be a general challenge across study fields.

Line 312-313: “A more recent study of the veterinary literature that focused on reporting adherence to the ARRIVE guidelines…”

Line 347-351: “We envisaged that an improvement in methodological reporting would be noticeable since many journals have endorsed the ARRIVE guidelines. However, advancements continue to progress at a slow pace or do not happen at all. We show that the reported information’s level of detail is generally incomplete. The incomplete reporting of these details directly impedes the ability to assess the validity of the experiments.”.

In regard to selection of specific Danish research institutions: These are the leading research institutions in Denmark reporting animal experimental research and of interest for this study.

Methods

Line 86: Was this estimation based on previous research or calculated in some way?

Response:

The estimation is based on pilot data from an internal study of published animal research from Aarhus University but primarily on group discussion taking into account the amount of time available for thorough and in-depth assessment of publications. It was estimated that 500 papers in total could be performed. Retrospectively, because we used simple random sampling, we assume that the remaining 1251 studies (3051-1800) contain 370 eligible studies (30 %). This assumption is based on the fact that of 1800 studies (identified by random sampling) screened for eligibility, we retrieved 531 eligible studies equalling 30%.

The sentence in line 86-88 now states: “It was estimated that a thorough assessment of 500 papers in total – 250 papers from each year – could be performed within the given timeframe.”.

Line 98: I see that the search strategy is based on the SYRCLE animal filter, but could it also be shown in full in the supplementary information?

Response:

Thank you for this comment. The syntax for the SYRCLE animal filter has now been included as a supporting information file (S1 File) and the search strategy is based on the search strategy found in reference number 35 and 36 in the manuscript.

Line 101: What are the 5 Danish universities and why were they selected?

Response:

This study include papers reporting animal experiments from Aarhus University (AU), Aalborg University (AAU), University of Copenhagen (UCPH), University of Southern Denmark (SDU) or Technical University of Denmark (DTU). This information is found in the supporting information S1 File. These are the leading research institutions in Denmark reporting animal experimental research and thus of most interest for this study.

Line 105: Inconsistencies in study number descriptions (written in text for 2009, then numerical values for 2018). These values would be better placed in the results section for clarity.

Response:

Thank you for noticing this. The sentence in line 105-107 is amended to: “One thousand, one hundred and sixty-one studies from 2009 and 1890 studies from 2018 were found.”.

In our opinion this is not part of the results but is part of the method used to reach the 250 papers from each year. It is part of figure 1 and we feel that it is very important to keep the figure in relation to the methods section for readers to more easily understand the flow. We have now mentioned this in the results section as well; in line 173: “The flow of the publications retrieved is described in the Methods section and in figure 1.”.

Response:

Thank you for this comment. We chose the Landis 4 criteria and a number of representative items of the ARRIVE guidelines. We found it more important in this study to investigate the level of detail and thus we ended up with 10 items in total. The Landis 4 is mentioned in the ARRIVE guideline. From our veterinary background we know that health status is a really important aspect of research which role is often neglected. The animals’ health or immune status can influence their physiology and behaviour as well as their response to treatments, and thus impact on experimental variability and scientific outcomes. However, the animals' health status or comorbidities before and during experiments is to our knowledge rarely considered, and therefore we thought that this would be interesting to investigate.

We have amended the sentence in line 133-138 to the following: “Each publication was assessed according to 10 items primarily based on the Landis four related to the quality of reporting of significant methodology and included in the ARRIVE guidelines [6, 25, 26]. The selection of items was due to the nature of the study capturing different types of animal research. One item “health status”, was chosen since it, to our knowledge, is scarcely investigated even though it may influence many research outcomes [22].”.

Results

Line 169: Repetition of prevalence being reported with 95% confidence intervals

Response:

Thank you for noticing. The sentence in line 169 “All prevalences are reported with 95% confidence intervals.” is deleted.

Line 155: Was this score (0-3) used for any summary statistics or analysis?

Response:

We did not use this data for summary statistics. We used the information to describe the information retrieved in a descriptive manner.

Line 182: for random allocation, the percentage values have been calculated based on the full 250 papers rather than the subset of applicable papers which seems incorrect

Response:

Thank you for noticing this. To make a fair judgement it was necessary to find out whether the studies could be randomized or not. However, studies not reporting randomization were quite challenging as it was often difficult to find out whether they could in fact be randomized or not due to unclear study design or groups. We therefore decided on the following:

a. Studies reporting randomization (randomized) or reporting that the study is non-randomized scored "yes, reported"

b. Studies not reporting randomization, however, randomization is possible score "no, not reported"

c. Studies not reporting randomization as randomization is not possible (non-randomized). We decided to score those studies “no, not reported” if nothing was reported about randomization, and write a comment that randomization is not possible for that specific study. One might argue that this is not completely fair. However, in our opinion these studies ought to report that randomization was not feasible for complete reporting and transparency.

We have discussed this in line 284-288: “In studies where blinding and randomization were not feasible, the reason why (e.g., study design) was rarely justified nor considered a limitation and acknowledged in the study report. A description of why such a precaution is not taken will bring the reader’s attention to the missing safeguard so the results can be judged accordingly”.

Response:

Thank you for your comment. We have calculated percent agreement and kappa as both have strengths and limitations, and as suggested in McHugh, M. L. (2012). Interrater reliability: the kappa statistic. Biochemia medica, 22(3), 276-282. We have kept our data in the supporting information S3 File and in addition commented on items reviewers did not always agree on.

In line 169-170 we have added the sentence: “Reviewer agreement and Cohen’s Kappa values are disclosed in the supporting information (S3 File).”.

Discussion

Response:

Thank you very much for this suggestion. Comparison of the current reporting status between five Danish universities was originally a part of our study and can be found in the supporting information S1 File. However, this was not feasible due to the experimental design and the procedure for random sampling, and we decided to look at this on a national level.

In addition, the following has been amended:

Line 20-21: The following sentence: “Using a predefined research protocol and a systematic search, we retrieved all relevant animal studies.” is edited to “We retrieved all relevant animal experimental studies using a predefined research protocol and a systematic search.”.

Line 30: “the method of random allocation” is edited to “random allocation method”

Line 61: “…fore example” is corrected to “for example”

Line 80: “…each of the reported items.” is amended to “...each reported item.”

Line 85-86 is corrected to “An equal number of studies from each year were included to compare the results between the two time periods.”

Line 87: “…and in depth…” is deleted.

Line 101: “…to…” is corrected to “…with…”

Line 110-112: “Due to the decision to perform a comprehensive search strategy to identify all relevant preclinical animal studies, the majority of the studies were not applicable.”.

Line 119: “Exclusion…” is corrected to “The exclusion…”

Line 132-133: “…to allow the assessment of reporting…” is amended to “and to assess reporting…”

Line 142: “…in addition to…” is corrected to “…and…”

Line 149 is amended to “…complete information defined in the item or where items were reported not conducted…”.

Line 152: “Annotations and quotes for each item…” is edited to “Each item’s annotations and quotes…”.

Line 154: “After completing the initial reporting quality assessments by each reviewer…” is edited to “After completing each reviewer’s initial reporting quality assessments…”.

Line 167: “table” is corrected to “tables”

Line 178: “presented” is corrected to “shown”

Line 194 and 196: “…the…” is added.

Line 224: “…that…” is deleted.

Line 232: “…the…” is added.

Line 247: “reason” is corrected to “reasons”.

Line 248: “…exact…” is corrected to “…precise…”.

Line 248: “reason” is corrected to “reasons”.

Line 259: “…to be present…” is deleted.

Line 262: Table 1 title is amended to “Table 1. Prevalence of reporting details in Danish preclinical research in 2009 compared to 2018.”.

Line 285: “…why…” is deleted.

Line 299: “Interestingly, successful randomization may be negatively influenced by small sample sizes…” is amended to “Interestingly, small sample sizes may negatively influence successful randomization…”.

Line 311: “rather” is deleted.

Line 317: “…then…” is deleted.

Line 318: “In general…” is deleted.

Line 328: “We found that several…” is amended to “Several…”.

Line 332: “…with…” is added.

Line 342-344: The sentence: “A fully disclosed health report based on a case-oriented approach to the FELASA (Federation of European Laboratory Animal Science Associations) guidelines should be mandatory [23, 24]. Has been amended to “A fully disclosed health report should be mandatory and based on a case-oriented approach to the FELASA (Federation of European Laboratory Animal Science Associations) guidelines [23, 24].”.

Line 351: “is unable to” is corrected to “cannot”.

Line 377: “be used as guidance on…” is corrected to “guide”.

Line 383: “…the…” is added.

We are looking forward to your reply.

On behalf of the authors.

Kind regards,

Dr. Kousholt

Attachment

Submitted filename: Response to Reviewers.docx

Click here for additional data file.^{(49.6KB, docx)}

PLoS One. doi: 10.1371/journal.pone.0275962.r003

Decision Letter 1

Gillian Currie

28 Jul 2022

PONE-D-21-20844R1Reporting quality in preclinical animal experimental research in 2009 and 2018: A nationwide systematic investigationPLOS ONE

Dear Dr. Kousholt,

Thank you for your efforts revising your manuscript in response to our suggestions.

My main concern is that your data should be made available- this is a requirement for publication. PLOS journals require authors to make all data necessary to replicate their study’s findings publicly available without restriction at the time of publication. The specific details can be found here: https://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions

I do not believe that a PRISMA checklist is necessary for this study but it is useful to include the graphical representation of the flow of studies (Figure 1).

I have made some further suggestions to changes to the text to help clarify meaning.

In addition, it is important to clarify how the reviewers were able to identify exploratory studies.

Please submit your revised manuscript by Sep 11 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

We look forward to receiving your revised manuscript.

Kind regards,

Gillian Currie

Academic Editor

PLOS ONE

Journal Requirements:

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

Additional Editor Comments:

Thank you for your efforts revising your manuscript in response to our suggestions.

I do not believe that a PRISMA checklist is necessary for this study but it is useful to include the graphical representation of the flow of studies (Figure 1).

Further suggestions to your specific text changes, to clarify meaning, are below:

I suggest that you remove the “Taking a step further” and instead “This study investigates the reported information's level of detail by assessing preclinical studies within all animal experimental research fields with one or more authors affiliated with Danish research institutions.”.

2. Line 49-52: “A prevalent hindrance in reproducing experiments and lack of translation to bedside is an unsatisfactory internal validity. It refers to the extent to which appropriate methodologies safeguarding against systematic errors (bias) are implemented in the design, conduct, and analysis of an experiment” Amend sentences to clarify meaning.

Response:

I do not think that the sentence “Internal validity refers to how appropriate methodologies safeguarding against systematic errors (bias) are implemented in the design, conduct, and analysis of an experiment” is accurate (this is not a definition of internal validity that I am familiar with). Amend to clarify e.g. define internal validity and then state that methodologies to safeguard against bias can be implemented to increase internal validity.

3. Line 74-75: “In this context, realization is necessary already at the planning stages to guarantee good reporting in the end.” Amend sentence to clarify meaning.

Response:

Thank you for this comment. We have clarified the meaning by rewriting the sentence:

I suggest that the wording is changed to clarify meaning. If I have understood your meaning correctly then I suggest the wording “In this context, the use of the ARRIVE guidelines by researchers is necessary at the planning stage to help improve experimental design and, in turn, improve reporting.”

4. Line 122: Please give details on how you defined and identified an exploratory study.

Response:

To clarify this we have added “…(i.e. studies investigating novel questions and hypothesis-generating studies)…”. The sentence now states:

I am still not clear as to how an exploratory study was identified. How did you decide it was a novel question? Or hypothesis-generating? Did the authors have to state this in their manuscript?

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

PLoS One. 2022 Nov 3;17(11):e0275962. doi: 10.1371/journal.pone.0275962.r004

Author response to Decision Letter 1

26 Aug 2022

Response to reviewers:

Dear Academic Editor, Dr. Gilllian Currie

Thank you for your continued interest in our paper (PONE-D-21-20844R1) and the effort that you have put into reviewing the revised manuscript. Please find our point-by-point reply below. In addition to this rebuttal letter labeled 'Response to Reviewers', we have uploaded a marked-up copy of the manuscript that highlights changes made to the original version labeled 'Revised Manuscript with Track Changes', and an unmarked version of the revised paper without tracked changes labeled 'Manuscript' as requested.

Response to Dr Currie’s comments and requests:

Thank you for your efforts revising your manuscript in response to our suggestions. My main concern is that your data should be made available- this is a requirement for publication. PLOS journals require authors to make all data necessary to replicate their study’s findings publicly available without restriction at the time of publication. The specific details can be found here: https://journals.plos.org/plosone/s/data-availability#loc-unacceptabledata-access-restrictions

Response:

Thank you for responding to our request. Time has passed and at this time point we are fully able to make all data necessary to replicate our study’s findings available. So it is no longer an issue for us to keep the data restricted. The datasets are currently stored in The Open Science Framework repository https://osf.io and thus ready to be made publicly available upon PLOS One’s request. Additionally, the Supporting information file, S3 File is renamed “S3 File: Agreement data” in line 501 and now only contains our agreement data.

I do not believe that a PRISMA checklist is necessary for this study but it is useful to include the graphical representation of the flow of studies (Figure 1).

Response:

Thank you for agreeing on this matter. We omitted the PRISMA checklist from the supporting information and maintained the graphical presentation of the flow of studies (Figure 1) in the manuscript and submitted the manuscript. We were then notified to include the checklist in order for the submission process to proceed and swiftly responded your decision in the “Enter comments” prior to re-submitting our manuscript. However, omitting the checklist resulted in a new “revisions sent back to author” notification with a request for the checklist. Thus, to resolve this and speed up the review process (currently > 1 year), we have included the PRISMA checklist and stamped it with “Not to be included in the manuscript”.

1. Line 72-78: “Previous research has investigated the prevalence of reporting of measures to reduce the risk of bias for specific animal disease models or subjects of interest [29-32]. Other previous evaluations of preclinical reporting have provided an overview of the reporting status of items related to the internal validity or rigor of these experiments (e.g. blinding and randomization) [33]. Taking a step further, this study investigates the reported information's level of detail by assessing preclinical studies within all animal experimental research fields with one or more authors affiliated with Danish research institutions.” I suggest that you remove the “Taking a step further” and instead “This study investigates the reported information's level of detail by assessing preclinical studies within all animal experimental research fields with one or more authors affiliated with Danish research institutions.”.

Response:

Thank you for this comment. The sentence is corrected according to your request. You will find the sentence: “This study investigates the reported information's level of detail by assessing preclinical studies within all animal experimental research fields with one or more authors affiliated with Danish research institutions.” in line 75-78.

2. Line 49-52:

Response:

We agree that the sentences indeed need an additional revision in order to be clear and concise and thank you for underscoring this fact. We suggest the following:

Line 43-46: “A prevalent issue is unsatisfactory internal validity. Internal validity is the extent to which a design and conduct of a study eliminates the possibility of systematic errors (bias) [4]. Appropriate methodologies safeguarding against systematic errors can be implemented in the design, conduct, and analysis of an experiment in order to increase the internal validity [4].”

3. Line 74-75: “In this context, realization is necessary already at the planning stages to guarantee good reporting in the end.” Amend sentence to clarify meaning. Response: Thank you for this comment. We have clarified the meaning by rewriting the sentence: Line 68-71: “The implementation may be hindered by the lack of engagement of multiple stakeholders who all must engage in improving the reporting quality. In this context, researchers’ use of the guideline is necessary already in the planning stages to guarantee good reporting.”. I suggest that the wording is changed to clarify meaning. If I have understood your meaning correctly then I suggest the wording “In this context, the use of the ARRIVE guidelines by researchers is necessary at the planning stage to help improve experimental design and, in turn, improve reporting.”

Response:

Thank you for this comment. We have clarified the meaning of the sentence by revising according to your suggestion: “The implementation may be hindered by the lack of engagement of multiple stakeholders who all must engage in improving the reporting quality. In this context, the use of the ARRIVE guideline by researchers is necessary already at the planning stage to help improve experimental design and, in turn, improve reporting.” in line 67-71.

4. Line 122: Please give details on how you defined and identified an exploratory study. Response: We defined an exploratory study as research connecting ideas to understand cause-effect and investigating novel relevant questions that have not previously been thoroughly studied. These studies do not test but generate hypotheses. To clarify this we have added “…(i.e. studies investigating novel questions and hypothesis-generating studies)…”. The sentence now states: Line 119-123: “The exclusion of studies was based on the following exclusion criteria: science related to farming, wild animals or invertebrates, environment, human (clinical) studies, in vitro research, not primary papers/publications, lack of abstract or full text, exploratory studies (i.e. studies investigating novel questions and hypothesis-generating studies), and studies containing no intervention or no Danish author affiliation.” I am still not clear as to how an exploratory study was identified. How did you decide it was a novel question? Or hypothesis-generating? Did the authors have to state this in their manuscript?

Response:

Thank you for your comment. Exploratory studies were identified through study author statements that the study was explorative, or the study was assessed to be hypothesis-generating and performed to e. g. investigate novel questions, develop models or investigate and validate new animal models, look for biomarkers, explore pharmacokinetics/biodistribution, gene identification/expression, or mapping of receptors or microcirculation”.

To clarify, we have amended the sentence in line 119-125: “The exclusion of studies was based on the following exclusion criteria: science related to farming, wild animals or invertebrates, environment, human (clinical) studies, in vitro research, not primary papers/publications, lack of abstract or full text, studies containing no intervention or no Danish author affiliation, and exploratory studies (the latter studies were identified through study author statements that the study was explorative, or studies were assessed to investigate novel questions and to be hypothesis-generating).”

We are looking forward to your reply.

On behalf of the authors.

Kind regards,

Dr. Kousholt

Attachment

Submitted filename: Response to Reviewers.docx

Click here for additional data file.^{(28.4KB, docx)}

PLoS One. doi: 10.1371/journal.pone.0275962.r005

Decision Letter 2

Gillian Currie

27 Sep 2022

Reporting quality in preclinical animal experimental research in 2009 and 2018: A nationwide systematic investigation

PONE-D-21-20844R2

Dear Dr. Kousholt,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Gillian Currie

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

PLoS One. doi: 10.1371/journal.pone.0275962.r006

Acceptance letter

Gillian Currie

12 Oct 2022

PONE-D-21-20844R2

Reporting quality in preclinical animal experimental research in 2009 and 2018: A nationwide systematic investigation

Dear Dr. Kousholt:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Gillian Currie

Academic Editor

PLOS ONE

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

S1 File. Protocol.

(DOCX)

Click here for additional data file.^{(43KB, docx)}

S2 File. Supplementary material.

(DOCX)

Click here for additional data file.^{(16.9KB, docx)}

S3 File. Agreement data.

(XLSX)

Click here for additional data file.^{(18.3KB, xlsx)}

S4 File. PRISMA 2020 checklist.

(DOCX)

Click here for additional data file.^{(32.4KB, docx)}

S1 Table. Data extraction form.

(DOCX)

Click here for additional data file.^{(15.2KB, docx)}

S2 Table. Criteria for the assessment of reporting quality.

(DOCX)

Click here for additional data file.^{(21KB, docx)}

Attachment

Submitted filename: Response to Reviewers.docx

Click here for additional data file.^{(49.6KB, docx)}

Attachment

Submitted filename: Response to Reviewers.docx

Click here for additional data file.^{(28.4KB, docx)}

Data Availability Statement

All data files are available from the Open Science Framework database: https://osf.io/7e829/.

[pone.0275962.ref001] 1.Baker M., 1,500 scientists lift the lid on reproducibility. Nature, 2016. 533(7604): p. 452–4. doi: 10.1038/533452a [DOI] [PubMed] [Google Scholar]

[pone.0275962.ref002] 2.Begley C.G. and Ellis L.M., Raise standards for preclinical cancer research. Nature, 2012. 483(7391): p. 531–533. [DOI] [PubMed] [Google Scholar]

[pone.0275962.ref003] 3.Voelkl B., et al., Reproducibility of animal research in light of biological variation. Nature Reviews Neuroscience, 2020. 21(7): p. 384–393. doi: 10.1038/s41583-020-0313-3 [DOI] [PubMed] [Google Scholar]

[pone.0275962.ref004] 4.van der Worp H.B., et al., Can animal models of disease reliably inform human studies? PLoS medicine, 2010. 7(3): p. e1000245–e1000245. doi: 10.1371/journal.pmed.1000245 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0275962.ref005] 5.Landis S.C., et al., A call for transparent reporting to optimize the predictive value of preclinical research. Nature, 2012. 490(7419): p. 187–91. doi: 10.1038/nature11556 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0275962.ref006] 6.Glasziou P., et al., Reducing waste from incomplete or unusable reports of biomedical research. The Lancet, 2014. 383(9913): p. 267–276. doi: 10.1016/S0140-6736(13)62228-X [DOI] [PubMed] [Google Scholar]

[pone.0275962.ref007] 7.Tikka C., et al., Quality of reporting and risk of bias: a review of randomised trials in occupational health. Occupational and Environmental Medicine, 2021. 78(9): p. 691–696. doi: 10.1136/oemed-2020-107038 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0275962.ref008] 8.Riley S.P., et al., A systematic review of orthopaedic manual therapy randomized clinical trials quality. The Journal of manual & manipulative therapy, 2016. 24(5): p. 241–252. doi: 10.1080/10669817.2015.1119372 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0275962.ref009] 9.Jerndal M., et al., A systematic review and meta-analysis of erythropoietin in experimental stroke. J Cereb Blood Flow Metab, 2010. 30(5): p. 961–8. doi: 10.1038/jcbfm.2009.267 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0275962.ref010] 10.Vesterinen H.M., et al., Improving the translational hit of experimental treatments in multiple sclerosis. Multiple Sclerosis Journal, 2010. 16(9): p. 1044–1055. doi: 10.1177/1352458510379612 [DOI] [PubMed] [Google Scholar]

[pone.0275962.ref011] 11.Macleod M.R., et al., Evidence for the Efficacy of NXY-059 in Experimental Focal Cerebral Ischaemia Is Confounded by Study Quality. Stroke, 2008. 39(10): p. 2824–2829. doi: 10.1161/STROKEAHA.108.515957 [DOI] [PubMed] [Google Scholar]

[pone.0275962.ref012] 12.Hirst J.A., et al., The need for randomization in animal trials: an overview of systematic reviews. PLoS One, 2014. 9(6): p. e98856. doi: 10.1371/journal.pone.0098856 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0275962.ref013] 13.Chalmers T.C., et al., Bias in treatment assignment in controlled clinical trials. N Engl J Med, 1983. 309(22): p. 1358–61. doi: 10.1056/NEJM198312013092204 [DOI] [PubMed] [Google Scholar]

[pone.0275962.ref014] 14.Schulz K.F., et al., Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. Jama, 1995. 273(5): p. 408–12. doi: 10.1001/jama.273.5.408 [DOI] [PubMed] [Google Scholar]

[pone.0275962.ref015] 15.Wood L., et al., Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study. Bmj, 2008. 336(7644): p. 601–5. doi: 10.1136/bmj.39465.451748.AD [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0275962.ref016] 16.Hróbjartsson A., et al., Observer bias in randomized clinical trials with measurement scale outcomes: a systematic review of trials with both blinded and nonblinded assessors. Cmaj, 2013. 185(4): p. E201–11. doi: 10.1503/cmaj.120744 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0275962.ref017] 17.Holman C., et al., Where Have All the Rodents Gone? The Effects of Attrition in Experimental Research on Cancer and Stroke. PLoS biology, 2016. 14(1): p. e1002331–e1002331. doi: 10.1371/journal.pbio.1002331 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0275962.ref018] 18.Button K.S., et al., Power failure: why small sample size undermines the reliability of neuroscience. Nat Rev Neurosci, 2013. 14(5): p. 365–76. doi: 10.1038/nrn3475 [DOI] [PubMed] [Google Scholar]

[pone.0275962.ref019] 19.Cressey D., UK funders demand strong statistics for animal studies. Nature, 2015. 520(7547): p. 271–2. doi: 10.1038/520271a [DOI] [PubMed] [Google Scholar]

[pone.0275962.ref020] 20.Ioannidis J.P.A., Why Most Published Research Findings Are False. PLOS Medicine, 2005. 2(8): p. e124. doi: 10.1371/journal.pmed.0020124 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0275962.ref021] 21.Baker D.G., Natural pathogens of laboratory mice, rats, and rabbits and their effects on research. Clinical microbiology reviews, 1998. 11(2): p. 231–266. doi: 10.1128/CMR.11.2.231 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0275962.ref022] 22.Mähler Convenor M., et al., FELASA recommendations for the health monitoring of mouse, rat, hamster, guinea pig and rabbit colonies in breeding and experimental units. Lab Anim, 2014. 48(3): p. 178–192. doi: 10.1177/0023677213516312 [DOI] [PubMed] [Google Scholar]

[pone.0275962.ref023] 23.Berset Convenor F., et al., Federation of European Laboratory Animal Science Associations recommendations of best practices for the health management of ruminants and pigs used for scientific and educational purposes. Lab Anim, 2021. 55(2): p. 117–128. doi: 10.1177/0023677220944461 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0275962.ref024] 24.Kilkenny C., et al., Improving bioscience research reporting: the ARRIVE guidelines for reporting animal research. PLoS Biol, 2010. 8(6): p. e1000412. doi: 10.1371/journal.pbio.1000412 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0275962.ref025] 25.Percie du Sert N., et al., The ARRIVE guidelines 2.0: Updated guidelines for reporting animal research. BMC Veterinary Research, 2020. 16(1): p. 242. doi: 10.1186/s12917-020-02451-y [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0275962.ref026] 26.Baker D., et al., Two Years Later: Journals Are Not Yet Enforcing the ARRIVE Guidelines on Reporting Standards for Pre-Clinical Animal Studies. PLOS Biology, 2014. 12(1): p. e1001756. doi: 10.1371/journal.pbio.1001756 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0275962.ref027] 27.Macleod M.R., et al., Risk of Bias in Reports of In Vivo Research: A Focus for Improvement. PLoS Biol, 2015. 13(10): p. e1002273. doi: 10.1371/journal.pbio.1002273 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0275962.ref028] 28.Ting K.H.J., Hill C.L., and Whittle S.L., Quality of reporting of interventional animal studies in rheumatology: a systematic review using the ARRIVE guidelines. International Journal of Rheumatic Diseases, 2015. 18(5): p. 488–494. doi: 10.1111/1756-185X.12699 [DOI] [PubMed] [Google Scholar]

[pone.0275962.ref029] 29.Gulin J.E.N., Rocco D.M., and García-Bournissen F., Quality of Reporting and Adherence to ARRIVE Guidelines in Animal Studies for Chagas Disease Preclinical Drug Research: A Systematic Review. PLOS Neglected Tropical Diseases, 2015. 9(11): p. e0004194. doi: 10.1371/journal.pntd.0004194 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0275962.ref030] 30.Avey M.T., et al., The Devil Is in the Details: Incomplete Reporting in Preclinical Animal Research. PLoS One, 2016. 11(11): p. e0166733. doi: 10.1371/journal.pone.0166733 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0275962.ref031] 31.Leung V., et al., ARRIVE has not ARRIVEd: Support for the ARRIVE (Animal Research: Reporting of in vivo Experiments) guidelines does not improve the reporting quality of papers in animal welfare, analgesia or anesthesia. PLOS ONE, 2018. 13(5): p. e0197882. doi: 10.1371/journal.pone.0197882 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0275962.ref032] 32.Macleod M.R., et al., Correction: Risk of Bias in Reports of In Vivo Research: A Focus for Improvement. PLoS Biol, 2015. 13(11): p. e1002301. doi: 10.1371/journal.pbio.1002301 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0275962.ref033] 33.Vries R.B.M., et al., A protocol format for the preparation, registration and publication of systematic reviews of animal intervention studies. Evidence-based Preclinical Medicine, 2015. 2(1): p. e00007. [Google Scholar]

[pone.0275962.ref034] 34.Leenaars M., et al., A step-by-step guide to systematically identify all relevant animal studies. Lab Anim, 2012. 46(1): p. 24–31. doi: 10.1258/la.2011.011087 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0275962.ref035] 35.Hooijmans C.R., et al., Enhancing search efficiency by means of a search filter for finding all studies on animal experimentation in PubMed. Laboratory Animals, 2010. 44(3): p. 170–175. doi: 10.1258/la.2010.009117 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0275962.ref036] 36.Covidence systematic review software. [cited 2020/ 11/12]; www.covidence.org.

[pone.0275962.ref037] 37.Hooijmans C.R., et al., SYRCLE’s risk of bias tool for animal studies. BMC Med Res Methodol, 2014. 14: p. 43. doi: 10.1186/1471-2288-14-43 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0275962.ref038] 38.Han S., et al., A checklist is associated with increased quality of reporting preclinical biomedical research: A systematic review. PLOS ONE, 2017. 12(9): p. e0183591. doi: 10.1371/journal.pone.0183591 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0275962.ref039] 39.Hair K., et al., A randomised controlled trial of an Intervention to Improve Compliance with the ARRIVE guidelines (IICARus). Research Integrity and Peer Review, 2019. 4(1): p. 12. [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0275962.ref040] 40.Smith C.J., et al., The immune system in stroke: clinical challenges and their translation to experimental research. Journal of neuroimmune pharmacology: the official journal of the Society on NeuroImmune Pharmacology, 2013. 8(4): p. 867–887. doi: 10.1007/s11481-013-9469-1 [DOI] [PubMed] [Google Scholar]

[pone.0275962.ref041] 41.Percie du Sert N., et al., Reporting animal research: Explanation and elaboration for the ARRIVE guidelines 2.0. PLOS Biology, 2020. 18(7): p. e3000411. doi: 10.1371/journal.pbio.3000411 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0275962.ref042] 42.Ioannidis J.P. and Khoury M.J., Assessing value in biomedical research: the PQRST of appraisal and reward. Jama, 2014. 312(5): p. 483–4. doi: 10.1001/jama.2014.6932 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0275962.ref043] 43.Smith A.J., et al., PREPARE: guidelines for planning animal research and testing. Lab Anim, 2018. 52(2): p. 135–141. doi: 10.1177/0023677217724823 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0275962.ref044] 44.Menon J.M.L., et al., The impact of conducting preclinical systematic reviews on researchers and their research: A mixed method case study. PloS one, 2021. 16(12): p. e0260619–e0260619. doi: 10.1371/journal.pone.0260619 [DOI] [PMC free article] [PubMed] [Google Scholar]

PERMALINK

Reporting quality in preclinical animal experimental research in 2009 and 2018: A nationwide systematic investigation

Birgitte S Kousholt

Kirstine F Præstegaard

Jennifer C Stone

Anders Fick Thomsen

Thea Thougaard Johansen

Merel Ritskes-Hoitinga

Gregers Wegener

Roles

Abstract

Introduction

Materials and methods

Study protocol

Selection of studies

Fig 1. Flow diagram describing the random sampling, screening, and selection of studies.

Data extraction and analysis

Results

Fig 2. Prevalence of reporting quality in Danish preclinical research in 2009 compared to 2018.

Table 1. Prevalence of reporting details in Danish preclinical research in 2009 compared to 2018.

Part I: The overall reporting status

Part II: Level of detail of reported items

Discussion

Supporting information

Acknowledgments

Data Availability

Funding Statement

References

Decision Letter 0

Gillian Currie

Roles

Author response to Decision Letter 0

Decision Letter 1

Gillian Currie

Roles

Author response to Decision Letter 1

Decision Letter 2

Gillian Currie

Roles

Acceptance letter

Gillian Currie

Roles

Associated Data

Supplementary Materials

Data Availability Statement

ACTIONS

PERMALINK

RESOURCES

Similar articles

Cited by other articles

Links to NCBI Databases