Are the Lives of Animals Well-spent in Laboratory Science Research? A Study of Orthopaedic Animal Studies in Turkey

Abstract

Background

As in all fields of medicine, animal studies are widely performed in orthopaedics and have increased in number over time. However, it is not clear to what extent these studies provide a basis for future research or advancements in clinical science. Concerns about the reliability and translational ability of animal studies have been reported, and major orthopaedic journals and organizations are encouraging the reduction of unnecessary experiments on animals.

Question/purposes

(1) What proportion of animal studies conducted for orthopaedic research in Turkey were never published? And of those that were published, how long did it take to publish? (2) What proportion of those studies were published in journals with an Impact Factor of 2 or more? (3) What proportion of those published papers were never cited or cited only once? (4) What was the contribution to science of an animal euthanized for orthopaedic research in Turkey?

Methods

We reviewed all oral and poster presentations at the Turkish National Congress of Orthopaedics and Traumatology from 2009 to 2017 (retrieved from the archives of Acta Orthopaedica et Traumatologica Turcica), as well as all postgraduate theses in orthopaedics from 1991 to 2017 (retrieved from the archives of the National Thesis Center of the Council of Higher Education) to identify all orthopaedic studies that involved animals. We searched the keywords “animal studies,” “experimental studies,” and “orthopaedics” in these archives. We defined animal research as orthopaedic studies based on animal models. From this search and using that definition, 252 studies were identified. Of those, 4% (9) were excluded as they were thesis studies with no abstract in the archives. Thus, a total of 243 animal studies performed in Turkey were included for analysis in this retrospective study. The abstracts of these studies were examined to determine the study model (such as bone fracture models, tendon healing models, cartilage models) and number of euthanized animals. Between 1991 and 2017, 9412 vertebrate animals were euthanized for these studies. We searched PubMed, Google Scholar, ResearchGate, and ORCID to determine whether these papers were subsequently published, in which journal, and how long after the initial presentation publication occurred. The Web of Science 2019 database was used to determine the Impact Factor of the journals, the total citation count of each study, and the mean annual citation for each study (citations per year). For purposes of this analysis, we divided journals into those with an Impact Factor of 2 or more, 4 or more, and those with an Impact Factor below 2. The mean annual citation per euthanized animal (citations per animal per year) was calculated to determine the contribution of a euthanized animal to science.

Results

A total of 42% (101 of 243) of the animal studies in Turkey were never published. For all published studies, the mean time to publication was 2.2 ± 2.6 years (95% CI 1.7 to 2.6). The proportion of studies published in orthopaedic journals with an Impact Factor of 2 or more was 14% (34 of 243). Among the 142 published papers, 38% (54) were either never cited or were cited only once, and the mean citations per year was 1.1 ± 1.7 (95% CI 0.7 to 1.3). The mean citations per animal/year among the 142 published studies was 0.03 ± 0.04 (95% CI 0.02 to 0.04).

Conclusion

In the 243 theses and national congress presentations, 9412 animals were euthanized. Based on the low percentage of papers using animals that were euthanized and the very low proportion of studies published in higher-Impact Factor journals or garnering more than a single citation, in aggregate, little seems to have been gained from the loss of animal life. Future studies should try to replicate or refute our results in other countries.

Clinical Relevance

Orthopaedic researchers should try to reduce their use of unnecessary animal studies, for example, by reporting on the use of the “3Rs” (replacement, reduction, and refinement) in the development of an animal study design, as well as through following universal guidelines so that a study might have a clinical impact. Researchers should not conduct an animal study until they are convinced that the expected results are quite likely to deliver substantial benefit to people or to advance science in a meaningful way; although this seems intuitive, our results suggest that this may not be taking place. Ethics committees in Turkey should consider more detailed questioning before approving animal studies. If our results are replicated elsewhere, then a broader look at how these approvals are conducted should be performed.

Introduction

As the dominant species on earth, humans have ruled over many life forms. Animals have been used for many purposes throughout history, with an evolving relationship initially founded on survival or companionship. But today, millions of animals are euthanized annually for scientific research [24], and many live in difficult conditions or are slaughtered to meet the increasing daily demands as pets, or for food or clothing. Although the use of animals in scientific research may seem reasonable given what might be less-noble uses and less-humane treatments in other settings (such as some agricultural settings or the entertainment industry), animal research studies are increasingly being questioned. There are serious concerns regarding adequate methodology [12] in animal studies and their ability to translate to humans [5]. Organizations such as the People for the Ethical Treatment of Animals allege that these studies are cruel. They remark on misleading animal models that cannot predict reliable effects in humans [15], and others have suggested that the cost of these experiments may outweigh their potential benefits [1]. In addition, although from the moral perspective, animals have the right not to be subjected to experiments that cause pain and death [7], laboratory animals do not have the choice of whether to be in a study. By contrast, the National Academy of Sciences has argued that none of the alternative methods for animal testing is sufficient to replace these studies [17]. Although there are many such alternatives [6] and more are being developed [8], many researchers assert that animal testing remains the best method for many applications, particularly in preclinical research [10].

More than 100 million animals are euthanized worldwide every year, and approximately 250,000 animals in Turkey are euthanized for scientific research [23, 24]. All this research is expected to be performed in accordance with the 3Rs rule (replacement, reduction, and refinement) [22] of animal research. Major orthopaedic associations such as the Orthopaedic Research Society expect members to follow these rules and other measures to prevent unnecessary pain, suffering, or distress [2]. Despite the attention that has been paid to this set of concerns, it remains unclear what proportion of animal research studies contribute meaningfully to science or provide a basis for further research. One possible way to assess this would be to determine how frequently (and how quickly) animal research is published after it has been completed and first presented, whether such studies are cited frequently, and whether they are published in journals with higher Impact Factors (which is another metric for citation efficiency). To begin to explore this, we evaluated these parameters in a set of orthopaedic animal studies in Turkey published over a long span of time to give us a perspective on the state of the science.

We therefore asked, (1) What proportion of animal studies conducted for orthopaedic research in Turkey were never published? And of those that were published, how long did it take to publish? (2) What proportion of those studies were published in journals with an Impact Factor of 2 or more? (3) What proportion of those published papers were never cited or cited only once? (4) What was the contribution to science of an animal euthanized for orthopaedic research in Turkey?

Materials and Methods

The data of orthopaedic animal studies performed in Turkey were collected from two main sources. All oral and poster presentations at the Turkish National Congress of Orthopaedics and Traumatology from 2009 to 2017 (retrieved from the archives of Acta Orthopaedica et Traumatologica Turcica and available from 2009 to today) were reviewed, in addition to all postgraduate theses in orthopaedics from 1991 to 2017 (retrieved from the archives of the National Thesis Center of the Council of Higher Education, available from 1991 to today) to identify all orthopaedic studies that involved animals. We defined animal research as the orthopaedic studies based on animal models, and searched these databases with the keywords; “animal studies”, “experimental studies” and “orthopaedics”. From this search and using that definition, 252 studies were identified from all the orthopaedic animal studies available in these archives. Of those, 4% (9) were excluded as they were thesis studies with no abstract available in the archives. Finally, we retrospectively analyzed 243 animal studies conducted in Turkey between 1991 and 2017, in which a total of 9412 vertebrate animals were euthanized (seven stray dogs, 20 goats, 37 sheep, 60 mice, 1514 rabbits and 7774 different types of rats).

We examined the abstracts of the 243 studies, and we recorded the study model (84 bone fracture models, 44 tendon healing models, 28 cartilage models, 87 other models) (Table 1), the dates of the study, and the number of animals used. To determine whether a study was subsequently published in a peer-reviewed journal, we comprehensively searched for publications in PubMed and Google Scholar using the study titles, keywords, and names of the researchers. In addition, the social network pages (ResearchGate and ORCID) and available curriculum vitae of the researchers were analyzed for publications. All researchers listed on the abstract were searched before classifying the studies as unpublished. We calculated the publication proportion of the studies as the ratio of the published studies divided by the total number of animal studies. The time to publication was defined as the time interval between the presentation of the abstract or date of the thesis and the publication date.

Table 1.

Experimental models and contribution of animals in orthopaedic studies

We used the Web of Science (The Clarivate Analytics, 2019) database to determine the Impact Factor of the journals in which the studies were published. We used the database of Scopus Scientific Journal Rankings when the journal Impact Factor was not listed in the Web of Science. For purposes of this analysis, we divided journals into those with an Impact Factor of 2 or more, 4 or more and those with an Impact Factor below 2. The journals were also classified as indexed if they were listed in the Science Citation Index Expanded collection of the Web of Science and unindexed if they were not.

We obtained the total citation count of the published articles from the date of publication to 2019 from the Web of Science database, and the mean citation count per year (citations per year) was calculated for each publication. To determine the contribution of each animal euthanized for orthopaedic research, we calculated the annual citations for each euthanized animal as citation per animal per year by dividing the citations per year by the number of euthanized animals in each study.

Descriptive statistics were analyzed using SPSS software version 20.0 (IBM Corp, Armonk, NY, USA). Nominal variables are stated as a frequency and percentage, and continuous variables are stated as the mean ± SD along with their 95% confidence interval.

Results

A total of 42% (101 of 243) of the orthopaedic animal studies in Turkey were never published. A total of 4440 animals were euthanized in the 101 studies that were never published. The mean time to publication for the 142 published studies was 2.2 ± 2.6 years (95% CI 1.7 to 2.6).

The proportion of studies published in orthopaedic journals with an Impact Factor of 2 or more was 14% (34 of 243 studies), and only 5% of the studies (11 of 243) were published in journals with an Impact Factor of 4 or more. The proportion of studies published in indexed journals was 48% (117 of 243), and 70 different journals published orthopaedic animal studies from Turkey.

Among the 142 published papers, 38% (54 of 142) were either never cited or were cited only once. A total of 963 citations were determined from the 142 published studies; 49% (470 of 963) from 34 studies published in journals with an Impact Factor of 2 or more and 23% (223 of 963) from 11 studies published in journals with Impact Factors 4 or more. The mean citations per year was 1.1 ± 1.7 (95% CI 0.7 to 1.3) for all published studies, 2.1 ± 2.8 (95% CI 1.2 to 3.1) for publications in journals with an Impact Factor of 2 or more , and 3.4 ± 4.5 (95% CI 0.3 to 6.4) for publications in journals with an Impact Factor of 4 or more.

The mean number of citations per animal per year in the studies that were published was 0.03 ± 0.04 (95% CI 0.02 to 0.04), and the citations per animal per year was 0.06 ± 0.07 (95% CI 0.03 to 0.07) for studies published in journals with an Impact Factor of 2 or more and 0.07 ± 0.08 (95% CI 0.02 to 0.12) for studies published journals with an Impact Factor of 4 or more.

Discussion

Animal studies should be acceptable only when they provide crucial and beneficial knowledge to humans or animals and whenever there is no alternative [16]. Although millions of animals are euthanized every year for scientific research [24], there are several concerns about the reliability of these experiments and their relevance as translational research that can improve human health [18, 21, 26]. Although the proportion of abstracts at major orthopaedic meetings that are later published has been studied [3, 9], the proportion of abstracts involving animal studies that eventually get published has not, to our knowledge, been studied despite the high numbers of animals used in this way. We found that nearly half of animal studies evaluated over a sustained period from one country did not get published in any peer-reviewed journal. Further, of those that were ultimately published, only 14% were published in journals with Impact Factors greater than 2, most of those studies garnered very few citations in other published studies, and the mean time-to-publication (of those that were published) was more than 2 years, and in some cases, it was much longer. In aggregate, we consider results like these to be, in general, an insufficient basis to sacrifice the lives of animals, and these results should cause us to rethink how we use these animals’ lives, and in every case where a scientists proposes to sacrifice an animal’s life in an experiment, to ask whether the sacrifice will genuinely advance science or improve human health in meaningful ways.

This study had several important limitations. These results were obtained only from Turkey; as such, it is possible that the results might only be generalizable to countries with relatively modest research infrastructure. Future studies should seek to confirm or refute these results, particularly in wealthier, larger nations with higher numbers of euthanized animals. Although our results might not generalize outside the borders of our country, the possibility that they might should cause other countries to evaluate this pressing ethical question. Another limitation is that we did not analyze study quality; Impact Factor is not a good surrogate for study quality, but it may be an adequate subsitute for scientific influence. Impact Factor only measures the number of citations divided by the number of scientific studies a journal publishes over a 2-year span, and so for the laboratory sciences, including animal experiments, we deemed it to be an acceptable way to decide whether a study would influence future research. For most of the studies that we examined in our study, it definitely did not, based on the low number of citations per publication. One other limitation of our study was that we did not evaluate commercialization of these studies. However, before commercialization of the results, animal studies must be subjected to further verification, similar to clinical research. Because many studies were not cited more than once, it is difficult to say whether these papers could contribute to an end-product that is beneficial to humans, but our preliminary findings suggest that this is unlikely.

Replacement, reduction, and refinement—the 3Rs of animal testing—should be considered by anyone intending to perform animal studies [20]. Allen et al. [2] reported the necessity of understanding the basic principles of the 3Rs to continue to use laboratory animals for orthopaedic research. Replacement refers to the necessity of an animal model and asks the researcher to use another study model such as a computational model or a human-based model. Refinement indicates good animal welfare in the laboratory for the animals to mimic their biologic environment and to feel less pain. Finally, reduction implies achieving the desired goal of the study with the minimum number of euthanized animals. The most important part of reduction is to design a study that is appropriately statistically powered. We note, though, that regardless of whether an animal study is adequately powered, and regardless of whether it finds a significant difference, it is important that it be published (or that the authors do their best to disseminate their results) [22]. Any data that are published—including no-difference studies [13, 14]—may then be subjected to a meta-analysis in the future and by publishing the results, researchers can avoid unnecessary duplication of the same experiments. Although the proportion of animal studies in this series that eventually were published was comparable to the proportion of the abstracts published after other major orthopaedic meetings overall [9, 27], the fact that so many animal studies go unpublished is a major ethical problem. The nonpublication of an animal study must be considered a serious failure, because, by definition, it means animal lives were spent but new knowledge was not disseminated. It is obvious that a number of our colleagues either are not making a sufficient effort to publish their research, are not asking questions that journals find sufficiently compelling, or are not designing animal studies that are sufficiently robust to answer the questions they posed. Any of these explanations results in an unethical waste of animal life. In this study, we identified 4440 animals that were euthanized in unpublished studies, and to us, this seems a terrible and unethical loss. We are not opposed to animal research or even to animals’ lives being used on behalf of humanity—we acknowledge that animals killed for food or clothing may provide some benefit for a human being—but these experimental animals were euthanized for no obvious gain. If the researcher is not confident that an animal study will be published, the experiment should not be conducted. The Turkish Society of Orthopaedics and Traumatology—and all societies where animal research is presented—should seriously consider asking about the implementation of the 3Rs rule for studies submitted for presentation to its meetings.

There is an encouraging effort in the orthopaedic community to overcome the poor methodology of animal studies and reduce those that are unnecessary. Journals, such as Clinical Orthopaedics and Related Research®, the Journal of Orthopaedic Research and recently the Journal of Bone and Joint Surgery, American Volume require authors to follow the Animal Research: Reporting of In Vivo Experiments (ARRIVE) guidelines for animal studies that are submitted to these journals [2, 4]. These guidelines were generated to improve the methodology of animal studies and reduce unnecessary and lower-quality animal studies over time [11]. Kilkenny et al. [12] reported that many publications derived from animal studies had insufficient methodology regarding experimental and statistical models. Most of the proceedings at the Annual Meeting of the American Academy of Orthopaedic Surgeons are published within 2 years [8], and the mean publication time of proceedings at the Turkish National Congress of Orthopaedics overall was reported to be approximately 12 months [3], although we found that it took more than 2 years to publish those animal studies that eventually were published, and more than four out of 10 in our study never were published. Therefore, researchers should not conduct an orthopaedic model in animals unless it follows the ARRIVE guidelines, and unless the study addresses an important gap in knowledge that obviously will advance science or improve human health (to maximize its likelihood of publication). We believe that all journals publishing animal research should require authors to adhere to the ARRIVE guidelines.

Animal studies are mostly performed in preclinical fields to examine the safety and adequacy of treatments and are expected to form a basis for future research for translation to humans. Although the National Academy of Sciences has reported that it is impossible to make advances in medical sciences without animal studies [17], the reliability and predictability of animal models are currently being widely questioned [18, 21, 26]. Because of these concerns about animal models, there have been several attempts to improve the translational ability of animal models in orthopaedics [19]. However, the current study has shown that a substantial number of animal studies were either never cited or cited only once. Hence, many of these studies by definition did not guide subsequent research, nor were they used as a foundation for projects that followed. The most common experimental design in the examined studies were fracture models, but these studies had a low publication percentage (56%), few citations, and the highest number of euthanized animal lives of the experimental models that we examined (Table 1). Therefore, researchers should consider more carefully the value of investigating the effects of many items such as herbal or traditional products on fracture healing in animals (as was the case in a number of the fracture model studies that we identified). There are many computational models that can be used to investigate several stages of fracture healing process, and there are ongoing developments of these models [25]. These computational models may provide a means by which to reduce the number of animal studies which would be consistent with the “replacement” of 3Rs rule. Considering that almost half of the animal studies based on fracture models were not even published, these experiments should not be performed unless there is a compelling rationale that the expected outcomes will be beneficial to another life. Otherwise, one must look into the animal’s eyes before euthanizing them, think again, and ask this question: Is an animal’s life worth 0.03 citations per year?

In conclusion, considering that such a high proportion of studies involving the deaths of animals were never published, so few were published in journals with a high Impact Factor, and so many of the published studies were not cited or cited only once, we conclude that apart from the education of the researchers doing the work, the lives of thousands of animals were spent for little return. Researchers should design animal studies keeping the 3Rs rule in mind and should follow the ARRIVE guidelines for publication of the findings. Ethics committees and orthopaedic associations should supervise the implementation of these rules in Turkey and if these results are seen in other countries, a broader review may be required for approving, supervising, or publishing animal studies.