What can we learn from retracted studies in the nursing field in the last 20 years? Findings from a scoping review

Abstract

Background and aim of the work:

Literature reviews have summarised the number of retracted studies and guidelines have been developed to prevent this issue. However, available data are scarce in the nursing field. Learning from other experiences may be able to increase awareness of the issue and prevent avoidable errors. Therefore, the intent of this study was to map retracted articles in the nursing field by investigating the reasons for retractions in order to elicit strategies to prevent their occurrence.

Methods:

A scoping review was performed by searching PubMed and Cumulative Index to Nursing and Allied Health (CINAHL) for articles published from 2001 to 2021. Quantitative primary and secondary studies related to the nursing field and written in English, with a “retracted article” message and/or presenting a retraction notice, have been included. The main reasons for retraction have been recorded, as well as the main features of the studies retracted.

Results:

Out of 274 studies, we detected 26 retractions, of which eight were literature reviews and seven were experimental studies. Editors were the most frequent party requiring retraction. The retracted studies originated from 11 countries and were mostly published (n = 19) in general nursing journals. Scientific misconduct was the main cause of retraction (n = 18), while the remaining retractions were due to other types of errors.

Conclusions:

Most of the study retractions were issued by editors and originated mostly from high-scientific output countries. Scientific misconduct represented the principal cause of retraction; from these failures, educational strategies have been identified in order to prevent issues and to increase awareness among researchers and healthcare professionals. (www.actabiomedica.it)

Introduction

Until 2000, the implications of the retraction of scientific publications aroused little attention, as evidenced by the lack of literature on the topic and due to the low incidence of retracted articles (1,2). Conversely, in recent years, retracted articles represent a growing problem in academic publishing, leading to several difficulties for editors, reviewers and authors in managing this issue (3). Several factors may underline the increase of the phenomenon, starting from the first retrospective review by Budd and colleagues (4) on the topic, and the further rise in awareness after the publication of the retraction guidelines by the Committee on Publication Ethics (COPE) (5). The reduction in barriers to publishing articles containing defects, in addition to the easier electronic access to sources, the greater critical appraisal of published studies, and the more extensive recommendations for retraction followed by faster retractions have been underlined as factors increasing the phenomenon (6). Even in the absence of ill-intention in incorrect published information, errors in articles may still have the potential to cause damage to patient care and public opinion, threatening patient safety and resulting in misleading beliefs regarding health treatments among the population (1,4). Examples of this can be seen throughout recent decades: the retraction in 2010 of Wakefield’s article published in 1998 on the association between the measles, mumps and rubella vaccine and autism represents one of the most widely discussed cases (7). Relevant examples of retraction can also be found in the recent literature, such as the retraction of an article on the use of hydroxychloroquine and chloroquine in the treatment of COVID-19 (8).

In the COPE guidelines, a “retraction” is defined as a mechanism to correct the literature and to advise the public of incorrect or flawed contents, and that the reported findings are unreliable as a result (5). In line with this definition, the retraction of a scientific article represents a mechanism to preserve the rigour, the integrity, and the trustworthiness of the scientific literature (5). Articles may be retracted for multiple reasons; of these, the most frequent are unreliable and incorrect information reported, plagiarism, lack of authorisation for using data, competing interest, privacy and ethical issues (5). However, the literature has reported contrasting findings regarding the main reason for retractions to date. A recent review of 330 retracted indexed articles reported that the reason for retraction was false, unreliable or fabricated data for 66.4% of analysed studies (8). Another study regarding the rehabilitation and sport field reported that the most common causes of retraction were honest errors or non-replicable results, while the least reported was redundant publication (9). In contrast, Fang and colleagues, after having reviewed more than 2000 articles indexed on PubMed, reported fraud or suspected fraud as the main reason for retraction in 43.4% of articles, with less frequent reasons including errors (21.3%), duplication (14.2%) and plagiarism (9.8%) (10). More recently, authors found that the most prevalent reason for retraction was the compromised peer review (44%) (1).

Other characteristics of retraction have emerged from recent reviews regarding the country of origin of the authors and the topic of retracted articles. Lievore and colleagues, found that 72.2% of articles were written by authors from the United States (US), suggesting that over represented countries as those with a greater history of research may see more retraction issues (8). Regarding the topics, retraction occurs across all disciplines, but seems to occur more frequently in the chemical, biomedical and life sciences (8). In the nursing research field, one recent review published in 2018 (including journals extracted from the Journal Citation Report) detected 29 retracted articles, mainly due to duplications (58% of the studies are written by US and Korean authors (11)). In this context, the increased pressures due to the “publish or perish” culture in academia (12), may trigger duplication or “salami slicing” publications.

Therefore, continuing to review and analyse the retractions in a given discipline over time can be seen as a strategy to document issues in research conduct, mainly in terms of research misconduct (13); on the other hand, as a strategy where documenting these issues might be a learning opportunity for PhD students and clinicians who are undertaking research projects, and who are pressed by publishing their works to achieve the expected career advancements. It may also be of value to those who have the responsibility to supervise and mentor the next generation of researchers, in order to understand the errors—even those that are unintentional—that can occur while doing research. In other words, detecting and reflecting on the reasons for retractions may be useful in learning how to promote research integrity (13). Learning from the experiences of others (14) may increase awareness of the issue and prevent avoidable errors. With this intention, we mapped retracted articles in the nursing field, investigating the reasons for their retraction to elicit strategies to prevent the future occurrence.

Methods

We conducted a scoping review according to the updated framework of Levac and colleagues (15,16), consisting of research question identification; detection and selection of relevant studies; charting of the data; and collection, summarisation and reporting of findings. In reporting the methods and the findings, we followed the Preferred Reporting Items for Systematic reviews and Meta-analysis extension-Scoping Reviews (PRISMA-ScR) (17) (Supplementary Table 1).

Supplementary Table 1.

Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) Checklist (17)

SECTION	ITEM	PRISMA-ScR CHECKLIST ITEM	REPORTED ON PAGE #
TITLE
Title	1	Identify the report as a scoping review.	1
ABSTRACT
Structured summary	2	Provide a structured summary that includes (as applicable): background, objectives, eligibility criteria, sources of evidence, charting methods, results, and conclusions that relate to the review questions and objectives.	1
INTRODUCTION
Rationale	3	Describe the rationale for the review in the context of what is already known. Explain why the review questions/objectives lend themselves to a scoping review approach.	2
Objectives	4	Provide an explicit statement of the questions and objectives being addressed with reference to their key elements (e.g., population or participants, concepts, and context) or other relevant key elements used to conceptualize the review questions and/or objectives.	4
METHODS
Protocol and registration	5	Indicate whether a review protocol exists; state if and where it can be accessed (e.g., a Web address); and if available, provide registration information, including the registration number.	Not Available
Eligibility criteria	6	Specify characteristics of the sources of evidence used as eligibility criteria (e.g., years considered, language, and publication status), and provide a rationale.	4
Information sources*	7	Describe all information sources in the search (e.g., databases with dates of coverage and contact with authors to identify additional sources), as well as the date the most recent search was executed.	4
Search	8	Present the full electronic search strategy for at least 1 database, including any limits used, such that it could be repeated.	4
Selection of sources of evidence†	9	State the process for selecting sources of evidence (i.e., screening and eligibility) included in the scoping review.	4-5
Data charting process‡	10	Describe the methods of charting data from the included sources of evidence (e.g., calibrated forms or forms that have been tested by the team before their use, and whether data charting was done independently or in duplicate) and any processes for obtaining and confirming data from investigators.	5
Data items	11	List and define all variables for which data were sought and any assumptions and simplifications made.	5
Critical appraisal of individual sources of evidence§	12	If done, provide a rationale for conducting a critical appraisal of included sources of evidence; describe the methods used and how this information was used in any data synthesis (if appropriate).	Not Available
Synthesis of results	13	Describe the methods of handling and summarizing the data that were charted.	6-7
RESULTS
Selection of sources of evidence	14	Give numbers of sources of evidence screened, assessed for eligibility, and included in the review, with reasons for exclusions at each stage, ideally using a flow diagram.	6-7
Characteristics of sources of evidence	15	For each source of evidence, present characteristics for which data were charted and provide the citations.	6-7
Critical appraisal within sources of evidence	16	If done, present data on critical appraisal of included sources of evidence (see item 12).	Not Available
Results of individual sources of evidence	17	For each included source of evidence, present the relevant data that were charted that relate to the review questions and objectives.	6-7
Synthesis of results	18	Summarize and/or present the charting results as they relate to the review questions and objectives.	6-7
DISCUSSION
Summary of evidence	19	Summarize the main results (including an overview of concepts, themes, and types of evidence available), link to the review questions and objectives, and consider the relevance to key groups.	7-10
Limitations	20	Discuss the limitations of the scoping review process.	9
Conclusions	21	Provide a general interpretation of the results with respect to the review questions and objectives, as well as potential implications and/or next steps.	9-10
FUNDING
Funding	22	Describe sources of funding for the included sources of evidence, as well as sources of funding for the scoping review. Describe the role of the funders of the scoping review.	Not Available

Research question identification

We identified two research questions: (a) What are the main characteristics of the retracted articles related to the nursing field over the past 20 years? and (b) What are the main reasons for article retractions in the nursing field?

Study identification and selection

Keywords such as retract* and nurs* were used to search PubMed and the Cumulative Index to Nursing and Allied Health (CINAHL). We included articles that met the following criteria: (a) observational studies, randomised controlled clinical trials, reviews, and single case studies; (b) reporting the messages of “retracted” articles and/or that studies that presented a retraction notice; (c) related to the nursing field; (d) written in English, Italian or Portuguese according to the languages accessible to the authors; and (e) published between the 1^st January 2001 and 1^st July 2021. We excluded publications that were not subject to retraction, studies for which abstracts could not be found, and studies that regarded “retraction” in a context different from publication retraction (e.g., “stoma retraction”), and retractions referring to conferences, posters and erratum publications.

Moreover, we excluded publications that used the term “withdrawn” in the retraction notices, as in the Cochrane publications (1,2,8,18). These publications were not affected by errors in the study process or resulting from misconduct, instead requiring an update of their contents or adaptions to new criteria.

Two researchers (see authors) independently assessed the eligibility (titles and abstracts) of the studies and a third researcher (see authors) was consulted to resolve any disagreements. An identical process was adopted for the full-text eligibility screening.

Charting the data

For each eligible publication, we extracted the following data: authors and the country of the first author, year of publication and that of retraction, title of the article and the journal of first publication, study design, title of the journal that issued the retraction, and reasons for retraction. A researcher (see authors) independently extracted the data using an Excel spreadsheet, and a second researcher (see authors) double-checked the input to ensure the quality of the process and the accuracy of the data.

Data collection, summarisation and reporting

Of the 26 retracted articles, 24 studies (list from authors) are still available online with the related notice of retraction also indicating the reason. The remaining two studies have been removed and replaced with the notice of retraction; therefore, 24 articles have been analysed for their main characteristics.

The retraction reasons were available for 24 out of 26 notices of retraction and have been classified into two main categories: (1) scientific misconduct, and (2) error(s), defined as follows:

1. Scientific misconduct was defined as “fabrication, falsification or plagiarism in proposing, performing and reporting research” (19) and meant as close to the concept of dishonesty (20). Scientific misconduct was further divided in to six subcategories, as follows: duplicate publication, data fabrication, data falsification, data truthfulness issues, unethical conduct, and plagiarism;

2. Error(s) was defined as a scientific process error (in designing, conducting or reporting the research, e.g., in extracting data), where it was not possible to identify intentional misconduct.

Descriptive statistics were used to count and present data as absolute values and percentage frequencies, or mean and median, alongside the standard deviation and interquartile range, respectively.

Results

Characteristics of articles retracted

Of 274 articles retrieved from the literature search, after removing the duplicates, 26 retracted studies were identified (Figure 1).

Figure 1.

Flowchart of studies included.

The time of retraction after publication varied from 0 to 9 years, with a median of 1 year (interquartile range, 1.00–4.25 years). The first five retractions occurred in 2007, and most of the studies were retracted in the last 10 years, with peaks of four articles in 2011 and 2021 (Figure 2).

Figure 2.

Number of articles retracted according to the year of retraction.

Among the 24 articles for which the full text is still available, eight (33%) were literature reviews, and seven (29.2%) had an experimental design, of which three adopted randomisation in the methods, and the others were observational (4, 16.7 %), clinical comparative (3, 12.5%), clinical longitudinal (1, 4.2%), and qualitative (1, 4.2%) studies.

Most retractions were requested or issued by the editor (17, 70.8%), while the remaining were by the authors of the studies retracted. The first authors of the retracted articles were from 11 different countries (Table 1), with the most represented being the United States of America (6, 25%), United Kingdom (5, 20.8%) and China (4, 16.7%). In 16 cases (66.7%), the article was written alongside other authors, who may originate from different countries, while 10 articles (31%) were written by a single author. Moreover, one author contributed to the retraction of three articles, and another author of two articles.

Table 1.

Distribution of the articles retracted according to the country of affiliation of the first author

Country of the first author	Retracted articles (N=26), n
Australia	3
China	4
Finland	1
Georgia	1
Iran	2
Nigeria	1
Pakistan	1
Saudi Arabia	1
Turkey	1
United Kingdom	5
United States of America	6

Legend. N, Number.

The 26 studies are spread among 19 journals (Table 2), mainly covering the nursing field in general (13, 50%), followed by specialist nursing journals (9, 34.6%) and non-nursing journals (4, 15.4%).

Table 2.

Distribution of retracted articles according to the publication journal

Journal	Retracted articles (N=26), n
Accident and Emergency Nursing	2
Aging Clinical and Experimental Research	2
American Journal of Infection Control	1
British Journal of Nursing	2
International Journal of Nursing Practice	2
International Nursing Review	2
International of Journal Mental Health Nursing	1
International of Journal Older People Nursing	1
Journal of Advanced Nursing	2
Journal of American Academy of Nurse Practitioners	1
Journal of Child and Adolescent Psychiatric Nursing	2
Journal of Clinical Nursing	2
Journal of Nursing Management	1
Journal of Paediatric Nursing	1
Journal of Professional Nursing	1
Journal of the American Medical Association - JAMA	1
Journal of the American Medical Directors Association	1
Nurse Education Today	1
Nursing in Critical Care	1

Legend. N, Number.

Reasons for article retraction

Of the 26 available notices of retraction, 24 clearly reported the reason for the retraction. Eighteen studies (75%) were retracted due to scientific misconduct, including seven articles (29.2%) retracted due to duplication and two articles (8.3%) due to plagiarism (Table 3). The remaining articles (6, 25%) were included in the error category. The detailed reasons for each retraction category are presented in Table 3.

Table 3.

Reasons of article retracting (n=24)

Reason of retraction	Retracted studies with reported reason N (%)
Scientific misconducting (N = 18; 75%)
Duplication of the article (significant overlaps with already published material)	7 (29.2)
Plagiarism of the article	2 (8.3)
Working with data without research team approval	2 (8.3)
Not all copyright permissions had been obtained	2 (8.3)
Discrepancy between the methods applied and the data presented	1 (4.2)
Manipulation of data	1 (4.2)
References could not be verified	1 (4.2)
Unauthorized use of data by the author and inability to recognize the contribution of others in the research	1 (4.2)
Incorrect use of data without ethical approval	1 (4.2)
Error (s) (N = 6; 25%)
Error in the process of designing the study and writing the manuscript regarding the presentation of the abstract	2 (8.3)
Error in the process of extracting data and communicating the results	1 (4.2)
Incorrect duplication of the article (unbeknownst to the authors similar article already published)	1 (4.2)
Internal disagreement of the institution of the author	1 (4.2)
Data programming error	1 (4.2)

Legend. N, Number.

Discussion and conclusion

We have designed and developed this scoping review with the intent to detect issues in nursing publications. Our aims was to review the literature for reflection in order to promote research integrity, which should be taught from undergraduate to doctorate levels and throughout the entire professional life of researchers (21).

Regarding the timeframe, we have considered the last 20 years in order to reflect the contemporary/current practice around study retractions (22). Comparing the period from 2001–2010 and 2011–2021, the number of retractions has increased in the last 10 years, in agreement with other reviews (1,8), likely as a consequence of ethical policies set by editors of the journals and published guidelines on retraction processes (5). More than one study/year has been retracted, mainly in the last decade, suggesting that nursing research, as in other fields (e.g., 8) have shown an increased risk of this issue; this should therefore be considered as an educational topic for preparing the future generation of scientists. Moreover, retractions seem to occur on a variable time scale after publication, in some cases closer, in others after 4.5 years. These findings are in line with previous reviews in the nursing field, which reported a median time range of 1 to 3.5 years (11). However, reviews in the physiotherapy and rehabilitation disciplines have shown a decline in publication retraction time in recent years, suggesting that in some fields these issues are detected early (3). Late retractions may have severe negative implications both in research and in practice, which should be investigated in the future according to the increased relevance of nursing care to patients and their outcomes (23).

The retracted publications differed in their study designs, with both single and multiple authors. This seems to suggest that all methods can lead to retractions, and that the presence of several authors who are expected to supervise each other and cooperatively detect issues in the manuscript does not eliminate the risk of retraction. Moreover, while the majority of retracted publications were singular episodes, five retracted studies were produced by two authors, suggesting that the same error or misconduct may be repeated over time. In general, retractions were mainly required by the editor, with only one-third of cases initiated by the authors; this seems to suggest that weaknesses are highlighted mainly from the sides of the journals and not due to self-scrutiny by the authors.

The retracted publications were published in a variety of scientific journals, mostly in general nursing journals, as reported by other reviews on different fields (e.g., 3). By considering the first author as a proxy for the country of origin of the retracted articles, our findings suggest that the phenomenon of retraction is global. The countries with the largest number of articles retracted are those that have a long history in nursing research, such as the United Kingdom and United States, as confirmed by Lievore and colleagues (8). The high number of retracted publications may be due to the higher number of scientific publications in these countries (18).

Reasons for retraction were categorised into two main issues: scientific misconduct and errors. The first is prevalent in the last 20 years: namely, the duplication of the article due to significant overlap with already published material, in line with previous reviews (1,18). According to Frankel (24), the research community is obligated to promote scientific integrity and oversight of research in order to control scientific misconduct. Above all, supervisors, senior researchers and educators should consider their responsibility to appropriately address these issues, both in preparing the next generation and also in playing their role as researchers by demonstrating exemplary conduct. Namely, the list of reasons that emerged may be considered as educational topics that merit discussion, such as how to avoid duplication and self-plagiarism; how to work in an effective manner in a research team by promoting reciprocal respect and offering critical appraisal skills as a source of research integrity; how to ensure that copyright has been acquired and obtained (e.g., while using research tools); and how to appropriately access ethical approval and analyse the data in a consistent way, for example by ensuring that designed analysis methods are applied and communicated when unexpected results are obtained. In this context, several tools, materials and sources can be used (25). However, stating the reason for the retraction is fundamental, allowing authors who have acted responsibly in informing the journal of problems with their work to not be stigmatised along with those who have made mistakes as a result of misconduct (2). In this regard, the British Medical Journal recently introduced a disclosure statement, by which the lead author has to confirm that the manuscript is honest. It will be interesting to see, in the future, if this editorial decision will affect the level of retractions in the journal (1).

Regarding the error category of reasons for retractions, the main causes were problems with data extraction or programming. Often these problems can occur as honest mistakes, although in some cases it can be difficult to establish whether there was an honest mistake or misconduct. Recent steps towards greater transparency and reproducible research have encouraged the sharing and deposition of data prior to publication, which may have an impact on reducing cases of retractions due to errors involving the data. In preparing data to be “ready for publication”, many problems can be detected and resolved before publication (1). In order to teach future generations of researchers how to prevent errors, multiple rounds of counterchecks, independent evaluations, and data analysis by the research team might help. Moreover, for both categories of reasons, acting as reviewers of journals, working with associate editors, and learning from and receiving feedback may increase awareness. Additionally, receiving effective feedback regarding the submission may also increase the authors’ skills in detecting and preventing errors.

This review has several limitations. First, only two search databases were considered according to the main intent of the scoping review. Retracted publications or retraction notices that were not labelled as “retraction” in the databases or that did not include the word “retraction” in their publication title may have been missed. Moreover, although the COPE guidelines (5) provide accurate recommendations on how a retraction notice should be drafted, we found non-uniform forms for this procedure delivered by authors and publishers. This might result in an underestimation or overestimation of reasons and also limits comparison with previous studies (26). Second, we have considered retractions to be related to the nursing field, which was mostly the case. However, not all publications were strictly related to nursing care. Third, considering the time interval from the publication of an article to its retraction and the incompleteness of the data for the year 2021, the number of retractions for this last year could be underestimated. Regarding geographical distribution of retracted articles, the country of the first author of the study was considered to indicate the origin country of the retraction, and this might have led to biases in attributing the country. Moreover, other relevant data on authorship has not been collected, as the number of authors, the country of all authors, and their position, as junior or senior researchers. Future investigations might consider these elements to develop a more comprehensive description of the phenomenon under study. In addition, detecting the funding received by authors to conduct the study could provide a source of further reflections.

In conclusion, publication represents the main form of official communication of the scientific community to spread knowledge to health professionals involved in the treatment process, aiming to ensuring that patients can access the greatest possible benefits in terms of prevention, diagnosis, care, and treatment. Compared to more established academic disciplines, the numbers of retraction in nursing are extremely low. However, it is unclear whether the increase retractions detected over time reflects an increase in the publication of flawed articles or, conversely, an increase in the detection of such articles. Eradicating misconduct is challenging and, to date, there is no uniform way in which institutions, journals or societies address this problem. Several strategies could be implemented from different perspectives and these should start with consistent education, supervision and mentorship of researchers. Educational changes, ensuring accuracy and transparency in research conducted in practice, as well as ethical collection and preservation of data and proof of research by promoting accurate systems of checks and peer-review involving research teams, are all encouraged. At a broader level, further efforts should be directed at ameliorating the peer-review processes, in tracing and accurately reporting the article retraction process, and in promoting the critical skills of health professionals to appraise published studies.

Conflict of Interest:

Each author declares that she has no commercial associations (e.g. consultancies, stock ownership, equity interest, patent/licensing arrangement etc.) that might pose a conflict of interest in connection with the submitted article.