Manuscript Referencing Errors and Their Impact on Shaping Current Evidence

Abstract

It is imperative that articles published in reputable peer-reviewed journals provide balanced, fair, objective, and accurate references. However, studies on the accuracy of references in various scientific disciplines demonstrate an error rate of 25%-54%. These errors can range from minor errors in citation accuracy to major errors that alter the original content and meaning of the material referenced. This article discusses importance of citation accuracy, reviews principles of good citation practices, and offers recommendations aimed to decrease citation error rates.

The pharmacy community relies on peer-reviewed articles in clinical and academic journals to maintain current knowledge and remain abreast of changes in specific areas of practice and pedagogy. Trust in the reliability of published literature is essential to a journal’s reputability. Central to substantiating an article’s findings are the introduction and discussion sections, which depend on the accurate review and summary of existing literature to frame new information. The overall expectation is that seasoned academicians and practitioners possess the requisite in-depth knowledge of their field and follow good practices in managing citations that both support and counter their research. However, several studies describe various issues that may arise when citing original research or reviews done by others.^1-3 While academics educate their students in how to properly cite research and reference appropriate sources, citation errors still occur in published manuscripts, even those written by expert researchers.

This is an important although not commonly discussed issue. Improper referencing can affect an individual article’s citation index, thereby exaggerating or diminishing the perceived impact of the authors’ research. This perceived impact can have several vital implications if this metric is used as a criterion for scholarship evaluation in promotion and tenure decisions. Another aspect is that erroneous or inaccurate quotations, if continually repeated, may turn inaccuracies into conventional knowledge.⁴ As an example, a 2018 review article published in Environmental Research debated the alleged role of heavy metals, such as mercury, in increasing the risk of autism spectrum disorders.⁵ The article cites another article that studied ethical controversies in published literature related to this subject.⁶ However, the citation for that article is inaccurate: the numerical information reported in the Environmental Research article differs from that given in the source article.⁶ More concerning (and perhaps ironic) is that the cited article was originally published online in 2015 and had been retracted for ethical conflicts and erroneous information in 2017.⁷ Nevertheless, it was cited in 2018 review in Environmental Research, thereby propagating potentially invalid and biased information.

As illustrated by this example, referencing errors have serious implications and can affect an author’s reputation. Furthermore, and perhaps more importantly, decisions that have an impact on patient care or the education of future health practitioners are often based on information found in the published literature. The responsibility lies with authors, peer reviewers, and editors to deliver high-quality research articles that are balanced and unbiased, and include a fair evaluation of how the new research findings fit within current knowledge.

Several guidance documents are available to direct proper referencing. The Office of Research Integrity (ORI) within the Department of Health and Human Services provides an excellent education module regarding the ethical principles of proper citations practices.⁸ Additionally, in 2002, in her insightful review of common referencing mistakes, Harzing proposed 12 straight-forward guidelines for good citation practices.¹ While her in-depth analysis offers a case study relevant to economics and commerce, these principles are easily applicable to other disciplines, including pharmacy practice and education. A literature search was conducted to identify relevant examples to support these principles. Reflections on some of the most important citation practices with applicable examples from the identified articles are presented below.

When working on a reference list, it is important to cite the correct references and provide complete and accurate information for each publication.¹ Several studies on the accuracy of references in various scientific disciplines demonstrate an error rate of 25%-54%.^2,9-11 Errors occur when there is incorrect or missing information, which hinders the identification of the correct reference. With wide use of reference management software applications, such as EndNote, Mendeley, or EasyBib, these types of errors are likely on the decline; however, they cannot be completely eliminated because of the author’s ability to manually enter some sources (such as conference proceedings, for example). Studies that evaluate the impact of these applications on referencing error rates are lacking.

Referring to an incorrect publication may occur when the citation may be accurate, but the intention was to cite a different article (eg, same author, different publication). Care must be taken to avoid this, as the software mentioned above would not be able to identify these types of errors.

Harzing proposed the term empty references¹ for publications that do not contain original research but rather refer to the original research done by other investigators. There are several problems with using empty references, for example, the increased opportunity for the introduction of erroneous information or misinterpretation. Copying references without reviewing the content of the articles and then offering an author’s own evaluation and interpretation of the findings is what Roig considers a “deceptive citation practice.”⁸ It is difficult to know when this occurs; however, reader’s confidence is violated if authors summarize and offer their opinion on issues discussed in articles that they have not read. Additionally, using seemingly updated “empty” references with a later publication year can create the illusion of newer data being available even though the cited reference still goes back to the older, original data.

Another important guideline is for an author to use reliable sources when they are considering adding a reference to an article’s citation list.¹ Peer-reviewed articles are a cornerstone of academic publishing. While other sources of information, such as trade journals, newspapers, non-peer-reviewed articles, and media can add to the narrative, using these sources to solely support a research claim is likely not appropriate. An author should also make sure that the references they cited had not been amended or retracted after publication, such as in the mercury example discussed earlier. Several studies indicate that retracted articles remain to be actively cited even after published retraction, with most citing articles including a positive overview of retracted research.^12-16

Using very specific data and extrapolating the data to conditions or populations not studied leads to irrelevant interpretation and unintended, incorrect implications, violating the “using generalizable sources for generalized statements” principle.¹ This also applies when using a small number of studies or studies of limited scope to imply generalizable conclusions.

Incorrect interpretation of the objectives, results, or implications of the referenced study is perhaps the most detrimental type of referencing error.¹ The ORI guidelines caution against using resources that are not thoroughly understood.⁸ When scientists across various disciplines were asked how well they knew the content of the references they cited in their research papers, 40% reported that they knew them only “slightly well” or “not well at all.” Additionally, scientists admitted being influenced by a referenced paper’s citation index, being less familiar with the details of prominent papers than with less well-cited papers. This finding demonstrates that authors sometimes rely on a citation index, which can serve as a proxy for a papers’ reputation, more than they rely on their own review and evaluation of the article (preliminary data).¹⁷

In their study of the accuracy of quotations and references in medical journals, de Lacey and colleagues found a 15% misquotation rate,⁴ a statistic that was confirmed by a more recent study by Mogull.³ The significance of these misquotes ranged from “trivial” to “seriously misleading,” with all six highly regarded journals included in this analysis having published from two to five seriously misleading quotations in 50 randomly selected articles.⁴ Similar rates of misinterpreted citations were reported in a study examining citations of review articles in the ecology journals.¹⁸

The problem of incorrect interpretation of the objectives, results, or implications published in a referenced study affects all journals, even the most reputable. In fact, while the intent of this article is not to evaluate misleading quotations in AJPE, this author’s own work was misquoted in several articles that were recently published in the Journal.^19,20 Comparing one’s study with the findings from other studies that had significantly different objectives would result in misinterpretation of the content of the referenced articles.

Citing contemporary references¹ is especially important in the field of healthcare, where advancements are fast-paced and the need to stay on top of the latest developments is self-evident. Of course, there are some landmark studies that may still be appropriate to cite. However, as mentioned before, using “empty” references to make a references list appear current is a discouraged practice.

Foregoing impartiality towards journals with high impact factors is another principle of good referencing practice.¹ Recognizing an author’s biases is important in addressing them, and the perceived contribution of the study to the field is one of the common biases. One study found that researchers’ perception of a study’s validity, significance, and generalizability were significantly influenced by its citation index.¹⁷

Finally, presenting balanced information, including potentially conflicting evidence or counter-evidence,¹ enriches the article and provides fair “food for thought” for readers. Searching for articles including the ones that support and others that contradict study findings should generate rich discussion points and provide objective points of view, considering all available evidence on the matter.

Recommendations

These recommendations to improve citation practices are directed at both authors and journals.² All authors (not just primary or corresponding authors) should review and approve all references; statements should be checked against the source manuscripts, and empty references should not be used.^2,21,22 To avoid incorrect implications, references should be cited next to the statement they support rather than grouped together at the end of the sentence or paragraph. Simultaneously, journal should require authors to provide an affidavit at the time of manuscript submission confirming that all references have been checked against the original works, along with full-text copies of all cited articles.^2,23

Journals should consider limiting the number of references permitted for various manuscript types. This will guide authors to select only the best references to support their statements, and give them more manageable ownership of their citations.^2,21,23 During the manuscript review process, editors and peer reviewers should conduct a random reference check to ensure citation accuracy.^24-26 The editorial office should convey to the authors that references will be evaluated and that accuracy is expected before a manuscript can be accepted for publication.

CONCLUSION

In summary, it is important to provide balanced, fair, objective, and accurate references that support research paper findings. Overlooking the principles of good practices in referencing can lead to disseminating incorrect information, inappropriately diminishing or exaggerating the author’s contributions to the field, and/or developing practices that, while being based on academic literature, are not based on good science.