Skip to main content
BMJ Open logoLink to BMJ Open
. 2022 Aug 2;12(8):e049421. doi: 10.1136/bmjopen-2021-049421

Spin within systematic review abstracts on antiplatelet therapies after acute coronary syndrome: a cross-sectional study

Audrey Wise 1,, Deepika Mannem 2, Wade Arthur 1, Ryan Ottwell 1, Benjamin Greiner 3, Derek Srouji 4, Daniel Wildes 4, Micah Hartwell 1,5, Drew N Wright 6, Jam Khojasteh 7, Matthew Vassar 1,5
PMCID: PMC9351322  PMID: 35918107

Abstract

Objectives

Spin is a reporting practice in which study results are misrepresented by overestimating efficacy or underestimating harm. Prevalence of spin varies between clinical specialties, and estimates are based almost entirely on clinical trials. Little is known about spin in systematic reviews.

Design

We performed a cross-sectional analysis searching MEDLINE and Embase for systematic reviews and meta-analyses pertaining to antiplatelet therapies following acute coronary syndrome on 2 June 2020. Data were extracted evaluating the presence of spin and study characteristics, including methodological quality as rated by A MeaSurement Tool to Assess systematic Reviews (AMSTAR-2). All data extraction was conducted in a masked, duplicate manner from 2 June 2020 to 26 June 2020.

Participants and setting

Not applicable.

Primary and secondary outcome measures

We assessed abstracts of systematic reviews on antiplatelet therapy following acute coronary syndrome and evaluated the prevalence of the nine most severe types of spin. We additionally explored associations between spin and certain study characteristics, including quality.

Results

Our searches returned 15 263 articles, and 185 systematic reviews met inclusion criteria. Of these 185 reviews, 31.9% (59/185) contained some form of spin in the abstract. Seven forms of spin (1, 2, 3, 4, 5, 7 and 9) among the nine most severe were identified. No instances of types 6 or 8 were found. There were no statistically significant relationships between spin and the evaluated study characteristics or AMSTAR-2 appraisals.

Conclusions

Spin was present in abstracts for systematic reviews and meta-analyses; subsequent studies are needed to identify correlations between spin and specific study characteristics. There were no statistically significant associations between spin and study characteristics or AMSTAR-2 ratings; however, implementing changes will ensure that spin is reduced in the field of cardiology as well as other fields of medicine.

Keywords: Myocardial infarction, Ischaemic heart disease, Adult cardiology, Coronary heart disease


Strengths and limitations of this study.

  • We strictly adhered to our protocol, which was published prior to data extraction on Open Science Framework. We made our protocol, data, analysis scripts, data extraction forms and other study artefacts available online to increase our transparency and reproducibility.

  • As recommended by Cochrane Collaboration, we used an independent group to verify our results and implemented both screening and data extraction in a masked, duplicate manner to strengthen our study and minimise bias.

  • The effect of spin on the reader’s opinions and medical recommendations could not be judged, and the degree of spin use may have affected readers differently. To help mitigate the subjectivity of spin, we used online coursework and training to prepare our team for screening and data extraction.

  • Our study had a cross-sectional design that did not allow for cause and effect analysis or generalisations to be made to other systematic reviews published in other journals.

  • Selection bias may have negatively influenced the data collection. However, this effect was reduced due to the extensive training on methodology and data collection performed prior to extraction.

Introduction

As the gold standard in evidence-based medicine, regulating bodies often use systematic reviews to develop clinical practice guidelines that physicians refer to for treatment decisions.1 2 Hence, the findings contained in systematic reviews and in their abstracts directly influence the care that patients receive and their prognosis.

The systematic reviews themselves serve as a means to collate all relevant primary data on a specific topic, critically appraise the literature, and condense the findings into a useful synthesis. A major issue for patient care is that the reviews often contain outcome reporting bias; that is, they share only positive findings, show selection bias, and/or transfer bias from primary data.3 This bias is further complicated by clinicians having limited access to full-text articles, as well as limited time to read them in decision-making contexts. A 2013 study, for example, found that nearly 75% of physicians read the abstract alone, rather than the full text of the article.4 Importantly, abstracts have been found to not always accurately represent the findings reported in articles. For example, Toma and colleagues reported that 41% of the abstracts of randomised controlled trials (RCTs) contained different efficacy estimates than the full text.5 Consistent with that report, other studies have shown that over 50% of abstracts had discrepancies in the results compared with the full text, and only 50% of abstracts discussed the side effects or harms experienced during clinical trials.6 7 Considering the importance of evidence-based medicine, the necessity for reducing bias, especially in the abstracts of systematic reviews, cannot be overstated.

One form of bias that is of increasing interest to systematic reviewers, clinical trial researchers and epidemiologists is spin. Spin involves misrepresenting research results to mislead readers into favouring a certain outcome. It has been identified in multiple studies, particularly in clinical trial research, accessible from these publicly available data sets.8 9

Accurate and objective research findings are necessary to guide patient care. For example, patients who have experienced acute coronary syndrome (ACS) with subsequent initiation of antiplatelet therapy are particularly at risk for morbidity and mortality arising from antiplatelet therapy inaccuracies. If clinicians misinterpret (or are misled by) clinical research on antiplatelet therapy, the results could be deadly.

However, no study has analysed spin in this body of literature. Thus, the primary objective of this study was to analyse the presence of spin in the abstracts of systematic reviews on antiplatelet therapy following ACS. The secondary objective was to assess the study characteristics, including journal impact factor, funding source and article type, associated with spin. We additionally describe methods for reducing spin in antiplatelet publications.

Methods

Search strategy

A systematic review librarian (DW) constructed the search strategies for the MEDLINE (Ovid) and Embase (Ovid) databases to locate systematic reviews and meta-analyses pertaining to antiplatelet therapies (figure 1). DW conducted these searches on 2 June 2020. Following the execution of these search strategies, the results were combined and uploaded into a screening platform, Rayyan (https://rayyan.qcri.org/). The duplicates were removed, and the titles and abstracts were independently screened for eligibility by two researchers (AW and DM).10 This process was performed in a masked, duplicate manner. Once the initial screening was complete, AW and DM resolved disagreements through discussion.

Figure 1.

Figure 1

Search queries.

Eligibility criteria

The following inclusion criteria had to be met for an article to be eligible for this study. The article must (1) be a systematic review with or without a meta-analysis; (2) focus on antiplatelet, anticoagulation or thrombolytic therapies following ACS; (3) be conducted on human subjects only; (4) be available in English; and (5) have an abstract. The following study types were excluded from our study: observational studies (case-control, cohort, surveys), clinical trials, narrative reviews and systematic reviews not related to antiplatelet therapies. We also excluded duplicates, withdrawn or retracted studies, letters to the editor, and any remaining article that did not meet the inclusion criteria. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocols (PRISMA) definitions of systematic reviews and meta-analyses were used.11 The selected studies were then uploaded to Stata for randomisation, and as long as the previous criteria were met, the first 200 articles were sequentially extracted.

Training

Several levels of training were completed to ensure the quality of this study. Prior to the start of the study, all researchers were trained on an overview of study designs in clinical research, deduplication, and screening on Rayyan, and publication of protocols using Open Science Framework (OSF).12 The two researchers who conducted the screening (AW and DM) also completed an online training course on systematic reviews and meta-analyses offered by John Hopkins University on the Coursera platform.13 Following the course, the researchers undertook 2 days of online and in-person training. The training session covered the definition and interpretation of the nine most severe forms of spin in systematic review abstracts as defined by Yavchitz et al.14 Finally, data extractors were trained on how to analyse a systematic review via A MeaSurement Tool to Assess systematic Reviews (AMSTAR-2). The AMSTAR tool was used to determine the methodological quality of each included systematic review and meta-analysis. A detailed outline of the training can be found in our study protocol.

Identification of spin and data extraction

Data extraction was completed by AW and DM, using a pilot-tested Google form to evaluate for nine types of spin. These forms of spin as defined in Yavchitz et al14 studies were chosen and settled on following a Q-sort survey with members of the Cochrane Collaboration. This consisted of method review and bias method group members from over 400 invited participants. Members ranked examples on the Q-sort survey for more severe and prevalent forms of bias and were evaluated in our study. The selection of the nine types of spin in our study is derived from the most severe types of spin Yavchitz el al14 identified as well as from publicly available data in recent published literature15 on spin for completeness. The following general information was extracted from all reviews: the intervention type (surgery, pharmacologic and non-pharmacologic); the year in which the review was received by the journal; the funding source (industry, private, public, none, not-mentioned and hospital); whether the review discussed adherence to PRISMA16 or PRISMA for abstracts17; whether the journal recommended adherence to PRISMA; and the journal’s 5-year impact factor. The nine types of spin are defined in table 1. Researchers coded each of the nine spin types as being either present or absent from the systematic review, except for type 4 spin if safety was not addressed in the manuscript. If a systematic review did not address safety, it was not included in the analysis for type 4 spin. Any disagreements about the presence of spin were reconciled between AW and DM, and if an agreement could not be reached WA and MV acted as mediators. Following spin data extraction, the quality of each systematic review was analysed using AMSTAR-2 (https://amstar.ca/).18 AMSTAR-2 has previously been used to evaluate systematic reviews in multiple medical disciplines, including surgery, cancer, psychiatry, pain management and general medical interventions.18–22 Based on scores from the 16-item scale, reviews were given confidence ratings of high, moderate, low or critically low.18 Inter-rater reliability scores (assessed by kappa) for AMSTAR-2 ranged from.31 to 1.0,18 with additional studies supporting its reliability, finding moderate inter-rater reliability (median K=0.56) among multiple raters. Construct validity coefficients were high (AMSTAR, r=0.91; Risk of Bias in Systematic Reviews, r=0.84).23 All study characteristics, spin data and AMSTAR-2 data were extracted in a masked, duplicate manner.14

Table 1.

Spin types and frequencies in abstracts (n=185)

Nine most severe types of spin No. (%), containing spin
(1) Conclusion contains recommendations for clinical practice not supported by the findings. 10 (5.4)
(2) Title claims or suggests a beneficial effect of the experimental intervention not supported by the findings. 2 (1.1)
(3) Selective reporting of or overemphasis on efficacy outcomes or analysis favouring the beneficial effect of the experimental intervention. 42 (22.7)
(4) Conclusion claims safety based on non-statistically significant results with a wide CI.* 6 (15.0)
(5) Conclusion claims the beneficial effect of the experimental treatment despite high risk of bias in primary studies. 6 (3.2)
(6) Selective reporting of or overemphasis on harm outcomes or analysis favouring the safety of the experimental intervention. 0 (0)
(7) Conclusion extrapolates the review’s findings to a different intervention (ie, claiming efficacy of one specific intervention although the review covers a class of several interventions). 1 (0.5)
(8) Conclusion extrapolates the review’s findings from a surrogate marker or a specific outcome to the global improvement of the disease. 0 (0)
(9) Conclusion claims the beneficial effect of the experimental treatment despite reporting bias. 3 (1.6)

*Safety not assessed in 145, n=40.

Statistical analysis

The overall frequency of spin, study characteristics and AMSTAR-2 ratings were analysed using descriptive statistics, and the results are reported as both frequency counts and percentages. Following the screening, we included 185 systematic reviews. Binary logistic regression was used to determine the strength of associations via ORs between the occurrence of spin in a study’s abstract with journal impact factor, study characteristics and AMSTAR-2 appraisal rating, and a multivariable model was used to control for multiple characteristics. Prior to the start of the study, we determined that 185 systematic reviews were needed to power the analyses using GPower V.3.1.9.7. Our analytic decisions are included in our protocol, and Stata V.16.1 (StataCorp, College Station, Texas, USA) was used for all analyses.

Patient and public involvement

This study did not include any human participants or public involvement and therefore did not meet the regulatory definition of human subject research per the US Code of Federal Regulations and was not subject to Institutional Review Board oversight.

Oversight, data availability, transparency, reproducibility and reporting

To ensure that our study is transparent and reproducible, we have made the protocol, extraction forms, data analysis scripts and other study artefacts available via OSF. Our data and analysis scripts were sent to an independent investigation team and reanalysed in a masked fashion to further the reproducibility. This study was conducted in tandem with other studies evaluating the presence of spin in systematic reviews in other medical fields. These studies adhered to a common methodology, which has been described elsewhere. While drafting this article, we integrated the relevant reporting guidelines from PRISMA24 and Murad and Wang’s25 guidelines for meta-epidemiological studies.

Results

General characteristics

Our search queries returned 15 263 articles and 12 913 remained after deduplication. Of these, 12 372 were excluded after the initial title and abstract screening, which left 541 unique systematic reviews. A full-text review of the articles further excluded 356 reviews. Finally, 185 systematic reviews and meta-analyses pertaining to antiplatelet therapies following ACS remained and were included in data extraction. Our exclusion list from the screening process with rationale can be found in figure 2.

Figure 2.

Figure 2

Preferred Reporting Items for Systematic Reviews and Meta-Analyses flow diagram of study selection. ACS, acute coronary syndrome.

All studies meeting inclusion criteria analysed pharmacological interventions. Of the 185 reviews, 61 (32.9%) systematic reviews were published in journals that recommended adherence to PRISMA, and 79 (42.7%) of the papers mentioned adherence to PRISMA or PRISMA extension for Abstracts (PRISMA-A). The majority of the systematic reviews and meta-analyses did not mention a funding source (93/185, 50.2%), followed by systematic reviews that did not receive any kind of funding (44/185, 23.8%). Of the papers that mentioned a funding source (92/185, 49.7%), the most common funding source was private funding (20/185, 10.8%), followed by public funding (17/185, 9.2%) and industry funding (11/185, 6%).

Spin in abstracts

Among the included systematic reviews, 31.9% (59/185) contained at least one form of spin in the abstract. Some articles contained more than one type of spin; therefore, 61 instances of spin were found. The most common form of spin found in the 185 articles was type 3 in 42 articles (42/185, 22.7%). Following type 3 spin, 10 articles (10/185, 5.4%) contained type 1 spin. Regarding type 4 spin, safety was not assessed in 145 systematic reviews. Therefore, this spin type was only evaluated in 40 of our included studies. Of those 40 studies, 15% (6/40) contained type 4 spin. Spin type 2 (2/185, 1.1%), type 5 (6/185, 3.2%), type 7 (1/185, 0.5%) and type 9 (3/185, 1.6%) were found less frequently. No abstracts contained type 6 or type 8 spin. There was no significance between spin and any of the evaluated study characteristics. All study characteristics in the presence of spin are mentioned in table 2.

Table 2.

General characteristics of systematic reviews and meta-analyses

Characteristics No. (%) of articles (n=185) OR (95% CI)
Total (%) Abstract contains spin Unadjusted Adjusted
Article mentions adherence to PRISMA
 No 106 (17.8) 36 (19.5) 1 (Reference) 1 (Reference)
 Yes 79 (42.7) 23 (12.4) 0.67 (0.34 to 1.32) 0.42 (0.16 to 1.07)
Publishing journal recommends adherence to PRISMA
 No 124 (67.0) 43 (23.2) 1 (Reference) 1 (Reference)
 Yes 61 (33) 16 (8.6) 0.80 (0.43 to 1.50) 0.65 (0.29 to 1.46)
Funding source
 Not funded 44 (23.8) 16 (8.6) 1 (Reference) 1 (Reference)
 Industry 11 (5.9) 4 (2.2) 1.00 (0.25 to 3.95) 1.27 (0.23 to 7.04)
 Not mentioned 93 (50.3) 26 (14.1) 0.68 (0.32 to 1.46) 0.62 (0.23 to 1.64)
 Private 20 (10.8) 7 (3.8) 0.94 (0.31 to 2.85) 0.86 (0.23 to 3.17)
 Public 17 (9.2) 6 (3.2) 0.95 (0.30 to 3.07) 0.96 (0.24 to 3.76)
AMSTAR-2 rating
 High 24 (13) 7 (3.8) 1 (Reference) 1 (Reference)
 Moderate 110 (59.5) 7 (3.8) 1.13 (0.32 to 3.98) 0.93 (0.23 to 3.87)
 Low 29 (15.7) 12 (6.5) 1.71 (0.54 to 5.41) 1.68 (0.44 to 6.49)
 Critically low 110 (59.5) 33 (17.8) 1.04 (0.39 to 2.75) 1.19 (0.38 to 3.77)
Journal impact factor*, M (SD) 7.80 (11.52) 6.90 (8.45) 0.99 (0.96 to 1.02) 1.01 (0.97 to 1.05)
Publication year (1991–2020) 1.05 (1.00 to 1.11) 1.09 (1.00 to 1.18)

*18 journals did not have an impact factor.

AMSTAR, A MeaSurement Tool to Assess systematic Reviews; PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses.

AMSTAR-2

After appraising the systematic reviews with the AMSTAR-2 instrument, we rated 110 (59.5%) of the 185 studies as ‘critically low,’ 29 (15.7%) as ‘low,’ 22 (11.9%) as ‘moderate,’ and 24 (13%) as ‘high’. We found no statistical significance between spin and AMSTAR-2 appraisals (table 3).

Table 3.

AMSTAR-2 Items and frequency of responses (n=185)

AMSTAR-2 item Response, n (%)
Yes No Partial yes
(1) Did the research questions and inclusion criteria for the review include the elements of PICO? 185 (100) 0 (0) 0 (0)
(2) Did the report of the review contain an explicit statement that the review methods were established prior to the conduct of the review and did the report justify any significant deviations from the protocol? 75 (40.5) 108 (58.4) 2 (1.1)
(3) Did the review authors explain their selection of the study designs for inclusion in the review? 151 (81.6) 25 (13.5) 9 (4.9)
(4) Did the review authors use a comprehensive literature search strategy? 55 (29.7) 42 (22.7) 88 (47.6)
(5) Did the review authors perform study selection in duplicate? 128 (69.2) 57 (30.8) 0 (0)
(6) Did the review authors perform data extraction in duplicate? 126 (68.1) 59 (31.2) 0 (0)
(7) Did the review authors provide a list of excluded studies and justify the exclusions? 6 (3.2) 51 (27.6) 128 (69.1)
(8) Did the review authors describe the included studies in adequate detail? 137 (74.1) 12 (6.5) 36 (19.4)
(9) Did the review authors use a satisfactory technique for assessing the risk of bias (RoB) in individual studies that were included in the review? 53 (28.6) 127 (68.6) 5 (2.7)
(10) Did the review authors report on the sources of funding for the studies included in the review? 7 (3.8) 178 (96.2) 0 (0)
(11) If meta-analysis was performed, did the review authors use appropriate methods for statistical combination of results?* 142 (76.8) 34 (18.4) 0 (0)
(12) If meta-analysis was performed, did the review authors assess the potential impact of RoB in individual studies on the results of the meta-analysis or other evidence synthesis?* 72 (38.9) 103 (55.1) 0 (0)
(13) Did the review authors account for RoB in primary studies when interpreting/discussing the results of the review? 74 (40.0) 111 (60.0) 0 (0)
(14) Did the review authors provide a satisfactory explanation for, and discussion of, any heterogeneity observed in the results of the review? 145 (78.4) 40 (21.6) 0 (0)
(15) If they performed quantitative synthesis did the review authors carry out an adequate investigation of publication bias (small study bias) and discuss its likely impact on the results of the review?* 110 (59.5) 66 (35.7) 0 (0)
(16) Did the review authors report any potential sources of conflict of interest, including any funding they received for conducting the review? 127 (68.6) 58 (31.4) 0 (0)

*Nine articles did not perform a meta-analysis, thus n=176.

PICO, this mneumonic is used to answer healthcare related questions and stands for P- patient, problem or population, I- intervention, C- comparison, control or comparator, and O- outcome(s).

Discussion

Our study found spin in approximately one-third (31.9%) of the included systematic reviews. Spin has been shown to be prevalent in RCTs across many different fields of medicine.9 26 Recent studies have shown spin to be present not only within RCT abstracts but also within abstracts of systematic reviews.27 28 The results of the present study support the findings by Austin et al which identified one-third to one half of systematic review abstracts contained at least one form of spin, noted in their publicly available data set.26 Although multiple studies have evaluated spin in systematic reviews, to our knowledge, ours is the first to evaluate the use of spin within antiplatelet research. Here, we highlight the most common and the most severe forms of spin we found and the influence they have on the clinical use of antiplatelet therapies.

The most common type of spin found among the systematic reviews and meta-analyses that we analysed was type 3 spin (selective reporting of or overemphasis on efficacy outcome or analysis favouring the beneficial effect of the experimental intervention). To further describe type 3 spin, if efficacy outcomes are specified within the full text and are not reported in the abstract, spin is committed. Consider the following example noted by Dundar.29 They examined mortality and major adverse effects associated with thrombolytic agents (alteplase, tenecteplase, streptokinase and reteplase) in the treatment of acute myocardial infarction. In addition to mortality, their outcome measures included bleeding, stroke, reinfarction, allergy and anaphylaxis. Of the six outcomes, only morality and stroke were reported in the abstract. Interestingly, three of the four outcomes that were not reported in the abstract (allergic reaction, anaphylaxis and reinfarction) showed no statistical significance. Due to omission of these non-significant harm outcomes, the efficacy outcome of decreased mortality was selectively reported. Selectively reporting efficacy outcomes while leaving out harm outcomes can lead physicians to believe that an intervention is safer than it is; this belief may potentially lead to preventable adverse outcomes.

The most severe type of spin (conclusion contains recommendations for clinical practice not supported by the findings) was identified in our sample. This form of spin had the second greatest frequency among the included systematic reviews in our study. A study by Kinnaird et al provides an example of this form of spin. The authors claim that ‘This study suggests that bivalirudin is more effective and safer than heparin monotherapy and should, therefore, be preferred over heparin monotherapy’.30 However, the recommendation for using bivalirudin over heparin was not supported by all outcomes of interest. Non-significant outcomes of interest included risk for stroke and myocardial infarction. Without consistent significant findings for all outcome measures, the claim for clinical practice is spin. In our view, this spin type is the most severe because a recommendation for clinical practice can have an immediate influence on patient care.

The methodological quality of the included systematic reviews was analysed using AMSTAR-2. Among the reviews in our sample, over half were scored as critically low based on the AMSTAR-2 grading criteria. AMSTAR-2 defines this category as having at least one critical flaw rendering the systematic review unable to provide an accurate summary or recommendation.18 However, the presence of spin in the abstract was not significantly associated with a low AMSTAR-2 score for the systematic review. In light of these findings, spin may be found in systematic reviews regardless of their quality; there may be a multitude of factors propagating spin aside from quality, including abstract word limitations, funding conflicts of interest and lack of author awareness of spin. Although improving methodological quality may not eradicate the presence of spin, our findings support an opportunity for future systematic reviews to increase the quality of publications via adherence to AMSTAR-2.

Our study illustrates a current gap in transparency within the research pertaining to antiplatelet therapies. Whether intentional or not, motivating factors for the use of spin within manuscripts include confirmation bias, allegiance bias, and the misrepresentation of results in hopes for publication. Ioannidis et al suggest that the motives include a belief that positive results are associated with an increased likelihood of publication and greater clinical significance.31 Regardless of whether it is intentional or not, the misrepresentation of data may lead to suboptimal patient care.32 To help improve the reporting quality of systematic reviews on antiplatelet therapies following ACS, we have a few recommendations. First, readers must be made aware that spin exists within systematic reviews, especially as physicians often use systematic reviews to develop clinical practice guidelines. Given that Bero et al identified in their publicly available data set that spin is less prevalent in publications evaluating spin,33 we believe an awareness of spin is vital to decreasing spin in publications. Thus, by creating an online resource dedicated to teaching spin, as well as presenting on spin at medical conferences, we believe knowledge of spin would be increased among medical students, residents and physicians and its prevalence in publications may decrease. Next, we suggest that journal editors require the implementation of spin screening prior to author submission. Our rationale is that entities who control article publications have the greatest influence to change the amount of spin present in publications. Furthermore, in an effort to de-escalate spin, we urge journal editors to strive for inferential reproducibility by using a second discussant to verify study findings prior to publication.31 Inferential reproducibility is attained for a publication by an independent reviewer re-analysing the study to see if the same results and conclusions are reached.34 Additionally, we recommend journal editors emphasise the publication of studies with neutral results given that researchers believe positive results are more likely to be published.34 Lastly, PRISMA and PRISMA-A were designed to improve the reporting quality of systematic reviews. However, our study did not find less spin reported in publications that adhered to PRISMA. Thus, we suggest the addition of an item regarding spin to PRISMA and PRISMA-A to help reduce spin present in publications and systematic reviews.

Strengths and limitations

Our study contained key strengths and limitations. Regarding strengths, we strictly adhered to our protocol, which was published prior to data extraction on OSF. We made our protocol, data, analysis scripts, data extraction forms and other study artefacts available online to increase our transparency and reproducibility. Furthermore, we used an independent group to verify our results and implemented both screening and data extraction in a masked, duplicate manner to strengthen our study and minimise bias. This process is recommended by the Cochrane Collaboration. Regarding limitations, spin by nature is subjective and interpretation could vary between extractors. First, the effect of spin on the reader’s opinions and medical recommendations could not be judged. The degree of spin use may have affected readers differently. To help mitigate the subjectivity of spin, we used online coursework and training to prepare the team for screening and data extraction. An additional limitation is that our study had a cross-sectional design that did not allow for cause and effect analysis or generalisations to be made to other systematic reviews published in other journals. Furthermore, selection bias may have negatively influenced the data collection. However, this effect was reduced due to the extensive training on methodology and data collection performed prior to extraction. Lastly, our searches may not have returned all systematic reviews on antiplatelet therapies following ACS although we conducted our searches using the largest bibliographic databases (MEDLINE and Embase).

Conclusion

In conclusion, our study found spin is present in abstracts of systematic reviews regarding antiplatelet therapies following ACS. Although there were no statistically significant relationships between spin and the evaluated study characteristics or AMSTAR-2 appraisals, this misrepresentation of research data poses a threat to optimal evidence-based clinical use of antiplatelet therapy following ACS. There are steps that can be taken to improve the quality of antiplatelet therapy systematic reviews.

Supplementary Material

Reviewer comments
Author's manuscript

Footnotes

Twitter: @audwise

Contributors: AW: acquisition, analysis and interpretation of data, manuscript drafting and revision. DM: acquisition, analysis and interpretation of data, manuscript drafting. WA: conception and design of the work, drafting and revision of manuscript. RO: conception and design of the work, revision of manuscript. BG: drafting of the manuscript. DS: revision of manuscript. DW: revision of manuscript. MH: analysis and interpretation of data, revision of manuscript. DNW: conceptual design of the work. JK: analysis and interpretation of data. MV: conceptual design of the work, revision of manuscript, guarantor.

Funding: This study was funded by the Oklahoma State University Center for Health Sciences Presidential Mentor-Mentee Research Fellowship Grant.

Competing interests: None declared.

Patient and public involvement: Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

Provenance and peer review: Not commissioned; externally peer reviewed.

Data availability statement

Data are available in a public, open access repository. Data are available upon reasonable request. To ensure that our study is transparent and reproducible, we have made the protocol, extraction forms, data analysis scripts and other study artefacts available via Open Science Framework.

Ethics statements

Patient consent for publication

Not applicable.

Ethics approval

Not applicable.

References

  • 1.Moher D, Shamseer L, Clarke M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev 2015;4:1. 10.1186/2046-4053-4-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Cook DJ, Greengold NL, Ellrodt AG, et al. The relation between systematic reviews and practice guidelines. Ann Intern Med 1997;127:210. 10.7326/0003-4819-127-3-199708010-00006 [DOI] [PubMed] [Google Scholar]
  • 3.Drucker AM, Fleming P, Chan A-W. Research techniques made simple: assessing risk of bias in systematic reviews. J Invest Dermatol 2016;136:e109–14. 10.1016/j.jid.2016.08.021 [DOI] [PubMed] [Google Scholar]
  • 4.Marcelo A, Gavino A, Isip-Tan IT, et al. A comparison of the accuracy of clinical decisions based on full-text articles and on Journal Abstracts alone: a study among residents in a tertiary care hospital. Evid Based Med 2013;18:48–53. 10.1136/eb-2012-100537 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Toma M, McAlister FA, Bialy L, et al. Transition from meeting Abstract to full-length Journal article for randomized controlled trials. JAMA 2006;295:1281–7. 10.1001/jama.295.11.1281 [DOI] [PubMed] [Google Scholar]
  • 6.Berwanger O, Ribeiro RA, Finkelsztejn A, et al. The quality of reporting of trial Abstracts is suboptimal: survey of major general medical journals. J Clin Epidemiol 2009;62:387–92. 10.1016/j.jclinepi.2008.05.013 [DOI] [PubMed] [Google Scholar]
  • 7.Fontelo P, Gavino A, Sarmiento RF. Comparing data accuracy between structured Abstracts and full-text Journal articles: implications in their use for informing clinical decisions. Evid Based Med 2013;18:207–11. 10.1136/eb-2013-101272 [DOI] [PubMed] [Google Scholar]
  • 8.Chiu K, Grundy Q, Bero L. 'Spin' in published biomedical literature: a methodological systematic review. PLoS Biol 2017;15:e2002173. 10.1371/journal.pbio.2002173 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Jellison S, Roberts W, Bowers A. Evaluation of spin in Abstracts of papers in psychiatry and psychology journals. BMJ Evid Based Med. [DOI] [PubMed] [Google Scholar]
  • 10.Murad MH, Wang Z. Guidelines for reporting meta-epidemiological methodology research. Evid Based Med 2017;22:139–42. 10.1136/ebmed-2017-110713 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Shamseer L, Moher D, Clarke M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ 2015;349:g7647. 10.1136/bmj.g7647 [DOI] [PubMed] [Google Scholar]
  • 12.Mannem D, Wise A, Arthur W. Evaluation of spin in the Abstracts of systematic reviews focused on antiplatelet therapies.
  • 13.Introduction to Systematic Review and Meta-Analysis | Coursera . Coursera. Available: https://www.coursera.org/learn/systematic-review [Accessed 13 Jun 2020].
  • 14.Yavchitz A, Ravaud P, Altman DG, et al. A new classification of spin in systematic reviews and meta-analyses was developed and ranked according to the severity. J Clin Epidemiol 2016;75:56–65. 10.1016/j.jclinepi.2016.01.020 [DOI] [PubMed] [Google Scholar]
  • 15.Ottwell R, Rogers TC, Anderson JM, et al. Evaluation of spin in the Abstracts of systematic reviews and meta-analyses focused on the treatment of acne vulgaris: cross-sectional analysis. JMIR Dermatol;3:e16978. 10.2196/16978 [DOI] [Google Scholar]
  • 16.Moher D, Liberati A, Tetzlaff J, et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 2009;6:e1000097. 10.1371/journal.pmed.1000097 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Beller EM, Glasziou PP, Altman DG, et al. PRISMA for Abstracts: reporting systematic reviews in Journal and conference Abstracts. PLoS Med 2013;10:e1001419. 10.1371/journal.pmed.1001419 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Shea BJ, Reeves BC, Wells G, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ 2017;358:j4008. 10.1136/bmj.j4008 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Li Q, Deng K, Jiang X. Methodological quality assessment of systematic review or meta-analysis using AMSTAR-2: the long-term effectiveness or efficacy of opioids for chronic non-cancer pain (Published Online First: 6 August 2019).
  • 20.Zhang H, Han J, Zhu Y-B, et al. Reporting and methodological qualities of published surgical meta-analyses. J Clin Epidemiol 2016;70:4–16. 10.1016/j.jclinepi.2015.06.009 [DOI] [PubMed] [Google Scholar]
  • 21.Goldkuhle M, Narayan VM, Weigl A, et al. A systematic assessment of Cochrane reviews and systematic reviews published in high-impact medical journals related to cancer. BMJ Open 2018;8:e020869. 10.1136/bmjopen-2017-020869 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Matthias K, Rissling O, Nocon M. Appraisal of the methodological quality of systematic reviews on pharmacological and psychological interventions for major depression in adults using the AMSTAR 2. Published Online First: 30 May 2019. [Google Scholar]
  • 23.Lorenz RC, Matthias K, Pieper D, et al. A psychometric study found AMSTAR 2 to be a valid and moderately reliable appraisal tool. J Clin Epidemiol 2019;114:133–40. 10.1016/j.jclinepi.2019.05.028 [DOI] [PubMed] [Google Scholar]
  • 24.PRISMA . Available: http://prisma-statement.org/PRISMAStatement/Checklist.aspx [Accessed 19 Jun 2020].
  • 25.Liberati A, Altman DG, Tetzlaff J, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med 2009;6:e1000100. 10.1371/journal.pmed.1000100 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Austin J, Smith C, Natarajan K, et al. Evaluation of spin within Abstracts in obesity randomized clinical trials: a cross-sectional review. Clin Obes 2019;9:e12292. 10.1111/cob.12292 [DOI] [PubMed] [Google Scholar]
  • 27.Nascimento DP, Gonzalez GZ, Araujo AC, et al. Eight in every 10 Abstracts of low back pain systematic reviews presented spin and inconsistencies with the full text: an analysis of 66 systematic reviews. J Orthop Sports Phys Ther 2020;50:17–23. 10.2519/jospt.2020.8962 [DOI] [PubMed] [Google Scholar]
  • 28.McGrath TA, McInnes MDF, van Es N, et al. Overinterpretation of Research Findings: Evidence of “Spin” in Systematic Reviews of Diagnostic Accuracy Studies. Clin Chem 2017;63:1353–62. 10.1373/clinchem.2017.271544 [DOI] [PubMed] [Google Scholar]
  • 29.Dundar Y, et al. Comparative efficacy of thrombolytics in acute myocardial infarction: a systematic review. QJM 2003;96:103–13. 10.1093/qjmed/hcg016 [DOI] [PubMed] [Google Scholar]
  • 30.Kinnaird T, Medic G, Casella G, et al. Relative efficacy of bivalirudin versus heparin monotherapy in patients with ST-segment elevation myocardial infarction treated with primary percutaneous coronary intervention: a network meta-analysis. J Blood Med 2013;4:129–40. 10.2147/JBM.S50595 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Avidan MS, Ioannidis JPA, Mashour GA. Independent discussion sections for improving inferential reproducibility in published research. Br J Anaesth 2019;122:413–20. 10.1016/j.bja.2018.12.010 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Boutron I, Altman DG, Hopewell S, et al. Impact of spin in the Abstracts of articles reporting results of randomized controlled trials in the field of cancer: the SPIIN randomized controlled trial. J Clin Oncol 2014;32:4120–6. 10.1200/JCO.2014.56.7503 [DOI] [PubMed] [Google Scholar]
  • 33.Bero L, Chiu K, Grundy Q. The SSSPIN study-spin in studies of spin: meta-research analysis. BMJ 2019;367:l6202. 10.1136/bmj.l6202 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Magoon R, Jose J. Safeguarding anaesthesia research from spin. Br J Anaesth 2020;125:e460–2. 10.1016/j.bja.2020.08.042 [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Reviewer comments
Author's manuscript

Data Availability Statement

Data are available in a public, open access repository. Data are available upon reasonable request. To ensure that our study is transparent and reproducible, we have made the protocol, extraction forms, data analysis scripts and other study artefacts available via Open Science Framework.


Articles from BMJ Open are provided here courtesy of BMJ Publishing Group

RESOURCES