Skip to main content
PMC home page
PLOS ONE logoLink to PLOS ONE
. 2020 Apr 17;15(4):e0230843. doi: 10.1371/journal.pone.0230843

Reporting quality and spin in abstracts of randomized clinical trials of periodontal therapy and cardiovascular disease outcomes

Murad Shaqman 1,#, Khadijeh Al-Abedalla 2,#, Julie Wagner 3, Helen Swede 4, John Cart Gunsolley 5, Effie Ioannidou 2,*
Editor: Dermot Cox6
PMCID: PMC7164582  PMID: 32302309

Abstract

Objective

Poor reporting in randomized clinical trial (RCT) abstracts reduces quality and misinforms readers. Spin, a biased presentation of findings, could frequently mislead clinicians to accept a clinical intervention despite non-significant primary outcome. Therefore, good reporting practices and absence of spin enhances research quality. We aim to assess the reporting quality and spin in abstracts of RCTs evaluating the effect of periodontal therapy on cardiovascular (CVD) outcomes.

Methods

PubMed, Scopus, the Cochrane Central Register of Controlled Trials (CENTRAL), and 17 trial registration platforms were searched. Cohort, non-randomized, non-English studies, and pediatric studies were excluded. RCT abstracts were reviewed by 2 authors using the CONSORT for abstracts and spin checklists for data extraction. Cohen’s Kappa statistic was used to assess inter-rater agreement. Data on the selected RCT publication metrics were collected. Descriptive analysis was performed with non-parametric methods. Correlation analysis between quality, spin and bibliometric parameters was conducted.

Results

24 RCTs were selected for CONSORT analysis and 14 fulfilled the criteria for spin analysis. Several important RCT elements per CONSORT were neglected in the abstract including description of the study population (100%), explicitly stated primary outcome (87%), methods of randomization and blinding (100%), trial registration (87%). No RCT examined true outcomes (CVD events). A significant fraction of the abstracts appeared with at least one form of spin in the results and conclusions (86%) and claimed some treatment benefit in spite of non-significant primary outcome (64%). High-quality reporting had a significant positive correlation with reporting of trial registration (p = 0.04) and funding (p = 0.009). Spinning showed marginal negative correlation with reporting quality (p = 0.059).

Conclusion

Poor adherence to the CONSORT guidelines and high levels of data spin were found in abstracts of RCTs exploring the effects of periodontal therapy on CVD outcomes. Our findings indicate that journal editors and reviewers should consider strict adherence to proper reporting guidelines to improve reporting quality and reduce waste.

Introduction

The abstract of randomized clinical trials (RCT) provides the reader with the first account of the trial objectives, methodology and results. Therefore, reporting accuracy, clarity and quality have a critical role during the initial assessment of the trial and affects the decision to read the full text [1]. Furthermore, in many geographic locations, RCT abstracts are often the only section of an RCT freely accessible to clinicians [2].

In recognition of the importance of RCT abstracts, the Consolidated Standards of Reporting Trials (CONSORT) for abstracts guidelines [3] were developed as an extension to the original CONSORT, addressing clarity, completeness and transparency and ensuring that key trial elements are properly reported. Hence, poor reporting refers to omitting important information in abstracts as required by the well-defined CONSORT items [2].

Furthermore, spin is defined as failure to accurately and faithfully report the findings of a scientific study in a manner that would affect the reader’s perception of the outcomes [4]. The tool for spin assessment in publication abstracts [4] identifies reporting practices that constitute an intentional or unintentional attempt to spin the results and/or conclusions leading to misreporting and bias. Despite the development of reporting and spin guidelines, abstracts in biomedical literature are often characterized by poor reporting quality and biased finding interpretation [5–12].

The impact of poor reporting and spin on the public and professional perception of research findings is discernible. In fact, abstracts with high levels of spin were found to be more frequently read compared to abstracts of the same trial after being edited to omit spin, and were also more likely to mislead clinicians to accept a clinical intervention as being beneficial despite a non-significant primary outcome [1]. Moreover, spin in abstracts percolates into media coverage and press releases, which in turn generates greater public attention [13] Paradoxically, articles that received greater media attention showed improved citation metrics in subsequent publications [14], creating what resembles of a vicious circle of public and scientific misinformation.

Ever since the publication of the earliest studies indicating a correlation between cardiovascular (CVD) disease and periodontitis [15, 16], the findings have received considerable professional and public interest. To test causal relationships, several RCTs explored the effect of periodontal therapy on CVD outcomes. Subsequently, the topic sparked intense debates between researchers, caused wide-scale media coverage and public interest, and prompted involved professional organizations to issue official statements [17, 18].

Although multiple periodontal-CVD RCTs have been published, the adherence to the CONSORT guidelines and the incidence of spin has not been studied. Therefore, the aim of this study was to evaluate the reporting quality and the incidence of spin in abstracts of RCTs investigating the effect of periodontal therapy on CVD disease outcomes.

Materials and methods

Search methods and study selection

Studies were retrieved from PubMed, Scopus based on search strategy shown below. In addition, we crosschecked 17 trial registration platforms included in the World Health Organization International Clinical Trials Registry Platform [19] to confirm trial registration status and information (S1 Table). The search was conducted for all registers on 01/01/2018.

Search keywords and limitations or filters for each database were as follows:

  1. Pubmed: ("Lipids"[Mesh] OR "Acute-Phase Proteins"[Mesh] OR "Blood Pressure"[Mesh] OR "Arterial Pressure"[Mesh] OR "Hypertension"[Mesh] OR "Hypotension"[Mesh] OR "Cholesterol"[Mesh] OR "Cholesterol, LDL"[Mesh] OR "Cholesterol, HDL"[Mesh] OR "Cholesterol Esters"[Mesh] OR "Embolism, Cholesterol"[Mesh] OR "Cholesterol, VLDL"[Mesh] OR "Cardiovascular System"[Mesh] OR "Cardiovascular Infections"[Mesh] OR "Cardiovascular Abnormalities"[Mesh] OR "Cardiovascular Diseases"[Mesh] OR "Cardiovascular Physiological Phenomena"[Mesh] OR "Endothelium"[Mesh] OR "Endothelial Cells"[Mesh]) AND ("Periodontal Debridement"[Mesh] OR "Periodontal Diseases"[Mesh] OR "Periodontal Pocket"[Mesh] OR "Alveolar Bone Loss"[Mesh] OR "Dental Scaling"[Mesh] OR "Periodontitis"[Mesh] OR "Dental Prophylaxis"[Mesh] OR "Periodontal Attachment Loss"[Mesh]) filter: clinical trial

  2. Scopus: TITLE-ABS-KEY (("Lipids" OR " Proteins*" OR "Pressure*" OR "Hypertension" OR "Hypotension" OR "Cholesterol" OR "Cardiovascular*" OR "Endothelium" OR "Endothelial*") AND ("Periodontal*" OR "Alveolar Bone Loss" OR "Dental*" OR "Periodontitis") AND "clinical trial") AND (LIMIT-TO (DOCTYPE, "ar"))

  3. 17 trial registration platforms (S1 Table) were searched. Since these platforms were limited to one or two keywords, “periodont*” was used as a main keyword, then records were scanned for eligible studies.

The retrieved articles were hand-screened for identification of additional RCT reports (Fig 1), and then duplicates were excluded.

Fig 1. Search Strategy with flow chart representing the identification, screening, eligibility and inclusion stages.

Fig 1

Open in a new tab

RCT report inclusion criteria

  1. Study Design:

    Only publications of periodontal-CVD RCTs were included. Cohort, non-randomized trials or observational trials were excluded. RCT publications in languages other than English were excluded.

  2. Participants:

    Targeted populations included adult patients with no systemic diseases other than CVD diseases. Studies were included if the participants were diagnosed with chronic periodontitis only. Studies with participants diagnosed with aggressive periodontitis, gingivitis, or peri-implantitis were excluded.

  3. Intervention:

    The tested intervention included subgingival scaling and root planing (SRP) or SRP with adjunctive therapy. Interventional studies employing adjuncts alone, supragingival scaling alone or surgical therapy were excluded.

  4. Outcomes:

    True or surrogate CVD outcomes were included. For descriptive purposes, outcomes were segregated into two groups [20]:
    1. CVD true events; such as angina, myocardial infarction, stroke, and CVD end points (CVD related-death).
    2. Surrogate outcomes; such as blood pressure, lipids, blood tests, ugh sensitivity C-reactive protein (hs-CRP), lipoproteins, and blood cell count.

Additional selection criteria for spin assessment

Only studies with a clearly defined primary outcome were included in the spin analysis. To fulfill this condition, the primary outcome should be either explicitly stated in the abstract or the full text. In cases were the primary outcome was not explicitly reported, the outcome stated in the sample size calculations was considered as the primary outcome. If no outcome was stated in the sample size calculation, we deduced a primary outcome based on the stated objectives of the study. If no primary outcome could be identified, the study was excluded. In addition, studies with multiple primary outcomes were excluded.

Data extraction and compilation

  1. Selection of studies was carried out according to the inclusion criteria. Titles and abstracts of the search results were initially screened for identification of other potential eligible studies. Then, full texts were retrieved and assessed to further assess eligibility. In cases of multiple published reports associated with the same trial registration number, the primary publication on the RCT results was selected to avoid duplication.

  2. Data extraction: Two authors (MS, KA) independently reviewed the abstracts -and the full texts, when needed- of the included RCT reports, and applied the CONSORT for abstracts [3] and the SPIN checklist [1]. Disagreements were resolved by a third author (EI). Two items of the CONSORT abstracts guidelines were excluded because they only apply to unpublished studies or conference abstracts.

  3. Characteristics of each RCT abstract and the respective publishing journal were extracted:
    1. Abstract word count
    2. Number of citations as shown on Scopus [21].
    3. Trial registration number, trial registration date was determined based on the information provided by the trial registry.
    4. Trial funding source, number of authors, geographic location,
    5. Trial sample size, intervention and outcomes.
    6. Journal metrics such as 5 years-impact factor, impact factor without self-citation, influence factor as reported on Thomson-Reuters/Clarivate Analytics 2018 [22].

  4. A decision-making guide was implemented to assist with the calibration and review process [23]. An Overall CONSORT Score for reporting quality and Overall Spin Score were calculated for each RCT publication based on the CONSORT and Spin checklist.

Statistical analysis

Cohen’s Kappa statistic was used to assess inter-rater agreement. For the descriptive analysis categorical variables were expressed as proportion percent. For the exploratory bivariate association between Overall CONSORT Score and related variables, we applied a Spearman correlation model. The limited sample size did not allow for a further multivariate regression model to assess predictors of reporting quality. Statistical analysis was performed with SPSS software (SPSS Inc).

Results

General findings

Twenty-four RCT reports were deemed eligible and entered the analysis (Fig 1). Among them, one study was a secondary analysis publication [24]. The PAVE study had multiple publications, and according to our inclusion criteria we included only the results publication [25]

For each trial, journal and article metrics are presented in Table 1.

Table 1. Characteristics of included publications and publishing journals.

RCT Year Journal metrics Article metrics
Journal 5 years impact Impact factor without self-citation Eigenfactor Influence factor Abstract Word count Authors # Citations Registration Geographic location Funding
[24] 2017 JCP 4.62 3.34 0.011 1.15 208 5 3 Yes Netherlands Yes
[25] 2009 JP 3.52 3.02 0.011 0.86 241 17 13 Yes USA Yes
[26] 2012 JCP 4.62 3.34 0.011 1.15 193 7 20 Yes Pakistan Yes
[27] 2015 Medicine 2.19 1.89 0.05 0.59 237 11 132 No Italy No
[28] 2014 JCP 4.62 3.34 0.011 1.15 174 4 198 No Brazil Yes
[29] 2015 JP 3.52 3.02 0.011 0.86 266 5 113 Yes Brazil Yes
[30] 2005 J Dent Res 5.72 5.07 0.02 1.55 160 5 40 No UK Yes
[31] 2006 Am Heart J 4.63 4.08 0.04 2.11 346 6 27 No UK Yes
[32] 2016 Clin Oral Invest 2.55 2.23 0.011 0.68 240 5 682 No China Yes
[33] 2007 S Med J 0.96 0.81 0.002 0.32 145 7 36 No Turkey No
[34] 2011 JP 3.52 3.02 0.011 0.86 239 3 12 No India No
[35] 2015 JP 3.52 3.02 0.011 0.86 277 4 1 Yes India No
[36] 2003 JCP 4.62 3.34 0.011 1.15 174 6 27 No UK Yes
[37] 2016 Lasers Surg Med 2.74 2.49 0.004 0.59 249 8 16 No Saudi Arabia Yes
[38] 2011 J Perio Res 2.71 2.70 0.004 0.63 253 5 1 No Jordan Yes
[39] 2014 Hypertension 6.74 6.16 0.04 2.19 254 13 40 Yes Australia Yes
[40] 2011 JCP 4.62 3.34 0.011 1.15 192 5 1 Yes China Yes
[41] 2012 JP 3.52 3.02 0.011 0.86 310 6 88 No Chile Yes
[42] 2010 Eur J Oral Sci 1.91 1.57 0.003 0.56 177 11 24 No Australia Yes
[43] 2007 NEJM 67.51 78.54 0.70 29.45 255 10 8 No UK Yes
[44] 2007 JCP 4.62 3.34 0.011 1.15 201 12 240 No Turkey Yes
[45] 2008 JCP 4.62 3.34 0.011 1.15 189 9 16 No Japan Yes
[46] 2009 JP 3.52 3.02 0.011 0.86 228 4 5 No Brazil Yes
[47] 2017 JP 3.52 3.02 0.011 0.86 252 10 25 Yes China Yes
Open in a new tab

JCP: Journal of Clinical Periodontology, JP: Journal of Periodontology, J Dent Res: Journal of Dental Research, Am Heart J: American Heart Journal, Clin Oral Invest: Clinical Oral Investigations, J Perio Res: Journal of Periodontal Research, Eur J Oral Sci: European Journal Oral Science, S Med J: Southern Medical Journal, Lasers: Surg Med: Lasers in Surgery and Medicine, NEJM: New England Journal of Medicine.

An overview of other outcomes of the included articles is shown Table 2.

Table 2. Descriptive data on primary outcome and interventions.

CHARACTERISTIC N = 24 %
1 Nature of primary outcome
True outcome (CV event) 0 0
Surrogate outcome 24 100
2 Primary outcome source
Explicitly stated in abstracts 3 13%
Explicitly stated in full texts 3 13%
Based on power analysis 7 29%
Implied in objectives 11* 33%
3 Intervention type
SRP alone 14 58%
SRP + adjunct 10 42%
Open in a new tab

* the outcomes of 11 papers were implied by objectives and were more than one primary outcome, they weren’t included in the spin analysis

Generally, all RCTs explored surrogate outcomes and none examined CVD events (Table 2). Only 3 abstracts had explicitly stated primary outcome, while for the remaining RCTs, we identified and extracted the primary outcome from the full text (Table 2). RCTs with more than one outcome were excluded from spin analysis, as primary outcome identification was impossible.

In terms of trial interventions, Table 2 shows that SRP alone was used in 58.3% of the included RCTs, while SRP and adjuncts were used in 41.7%. The majority of the RCTs reported some type of funding (83.3%). Specifically, 58% were funded intramurally, 46% by foundations, 29% by federal agencies, and 25% by industry. 60% of the funded RCTs reported multiple funding sources.

Abstract word count for each trial is presented in Table 1, and was categorized into 3 groups according to CONSORT findings [3], which is presented in Table 2. 66.7% of the abstracts had <250-word count, 25% had word count from 250–300, and 8.3% had word count >300 [3].

CONSORT checklist findings

Following the RCT abstract assessment, Overall CONSORT Score ranged from 2 to 9 out of 15. Table 3 presents the frequency of each CONSORT item fulfillment.

Table 3. Fulfillment (%) of CONSORT items stratified by CONSORT sections.

CONSORT FOR ABSTRACT CHECK LIST (NUMBER) PERCENTAGE
TITLE 12 (50%)
TRIAL DESIGN 3 (13%)
METHODS
    PARTICIPANTS 0.0%
    INTERVENTIONS 13 (54%)
    OBJECTIVE 22 (92%)
    OUTCOME 3 (13%)
    RANDOMIZATION (METHOD) 0 (0%)
    BLINDING (MASKING) 0 (0%)
RESULTS
    NUMBERS RANDOMIZED 20 (83%)
    NUMBERS ANALYZED 4 (17%)
    OUTCOME 3 (13%)
    HARMS 1 (4%)
CONCLUSIONS 3 (13%)
TRIAL REGISTRATION 3 (13%)
FUNDING 8 (33%)
Open in a new tab

Specifically, only 50% of the included RCTs acknowledged the term “randomized clinical trial” in the title. Only 3 of the studies (13%) included the specific study design in the abstract (i.e. parallel group, crossover, superiority, etc.).

Assessment of methods reporting

Three items in the methods section lacked reporting in all RCTs, including the item “participants”, which lacked information about the location of the study and the detailed description of the participants that were included, and the item “randomization”, were all of the studies did not report the randomization method that was used, and the item “blinding”, were studies did not report the level blinding. The item “intervention” mostly lacked the necessary detailed description of the intervention; therefore, only 54% of the RCTs fulfilled this item.

Most of the study abstracts (92%) included the objectives of the study. Interestingly, only 3 abstracts (18.5%) explicitly stated the primary outcome.

Assessment of results reporting

Although 83% of the abstracts included the numbers of randomized populations, only 17% included the numbers of analyzed populations as part of the abstract materials and methods rather than the results section violating the CONSORT recommendations. Only 1 RCT indicated the harms in the abstracts (4%).

Assessment of conclusion, registration and funding reporting

Only 13% discussed the results and conclusion of the primary outcome. Trial registration information was reported in only 3 abstracts (13%). When the public trial registration records were examined using the registration number included in the study, we found that 4 RCTs registered following the study initation and the first subject recruitment.

Spin analysis findings

After applying the exclusion criteria as outlined in the methodology, 14 out of the 26 RCT reports were included in the spin analysis (S2 Fig). The prevalence and type of spin for the included articles is outlined in Table 4.

Table 4. Fulfillment (%) of SPIN items in result and conclusion sections.

Type of spin (number) Percentage
1) spin in the result
Focus on statistically significant within-group comparison 3 (21%)
Focus on statistically significant secondary outcomes 9 (64%)
Focus on statistically significant subgroup analyses 2 (14%)
Focus on statistically significant modified population of analyses (eg, per-protocol analyses) 4 (29%)
Focus on statistically significant within- and between-group comparisons for secondary outcomes 9 (64%)
OTHER SPIN: NO DEFINITION OF PRIMARY OR SECONDARY OUTCOMES 11 (79%)
2) spin in the conclusions
Focus only on treatment effectiveness:
1. Claiming equivalence for statistically nonsignificant results 0 (0%)
2. Claiming efficacy with no consideration of the statistically nonsignificant primary outcome 9 (64%)
3. Focusing only on statistically significant results 6 (43%)
Acknowledge statistically nonsignificant results for the primary outcome but emphasize the beneficial effect of treatment 9 (64%)
Acknowledge statistically nonsignificant results for the primary outcome but emphasize other statistically significant results 9 (64%)
Other spin in Conclusions section:
1. Conclusion ruling out an adverse event on statistically nonsignificant results 0 (0%)
2. Conclusion focusing on within-group assessment (both treatments are effective/treatment administered in both groups is effective (eg, add-on studies) 2 (14%)
3. Recommendation to use the treatment 1 (7%)
4. Focus on another objective 1 (7%)
5. Comparison with placebo group of another trial 0 (0%)
6. Statistically nonsignificant subgroup results reported as beneficial 0 (0%)
Others: Inadequate extrapolation to larger population, intervention or outcome 12 (86%)
misleading statements designed to exaggerate or falsely claim efficacy 7 (50%)
3) Spin in both results and conclusions 12 (86%)
Open in a new tab

Some form of spin in both of the results and conclusions sections was detected in the majority of the RCTs (86%). Given that 79% of the included studies failed indicate the primary or secondary outcomes in the abstracts, we considered that these studies employed diverse strategies of spin.

In the results section, at least one checklist item showed a form of spin. 64% of the studies focused on statistically significant secondary outcomes, and on statistically significant within- and between- group comparisons of secondary outcomes.

In the conclusion sections, half of the included RCTs (50%) made statements that were misleading and designed to exaggerate or falsely claim efficacy. 43% of the conclusions were focusing only on significant results regardless if they corresponded to the primary outcome, 7% focusing on another objective, and 7% making treatment recommendations. 64% acknowledged statistically non-significant results for the primary outcome yet emphasized the beneficial effect of treatment, and emphasized other statistically significant results.

Bivariate correlation analyses

The Overall CONSORT Score ranged between 2–9 out of 15 with some articles fulfilling only had 2 items of the CONSORT checklist. The maximum number of fulfilled items by a single publication was 9 out of 15 (S1 Fig). The Overall Spin Score ranged between 1–13. Some publications included only 1 item with some form of spin, while some articles had 13 items that were spun (S2 Fig).

Within the limitations of the study, there was a positive and significant bivariate correlation between the Overall CONSORT Score and funding source (correlation coefficient: 0.416, P-value: 0.043). In addition, we observed a significant correlation between Overall CONSORT Score and registration reporting in the abstract (correlation coefficient: 0.518, P-value: 0.009). In summary, we observed that abstracts that included trial registration and funding information were characterized by higher reporting quality.

Overall Spin Score showed a marginal negative correlation with Overall CONSORT Score (correlation coefficient: -0.517, P-value: 0.059), which signified that the higher the reporting quality in the abstract, the lower the spin. Overall Spin Score showed negative correlation, although not significant, with funding and registration. Overall Spin Score showed marginal negative correlation with the number of publication authors (correlation coefficient of -0.509, P-value of 0.063).

Discussion

Our study evaluated the reporting quality and incidence of spin in the abstracts of 24 RCT publications assessing the impact of periodontal interventions on CVD outcomes. To our knowledge, this is the first paper that evaluated both reporting quality and spin in abstracts of such publications. The overall reporting quality of the included abstracts was deemed to be poor. Overall, we found that the RCT objectives and numbers of randomized subjects were the only most adequately reported items (92% and 83% respectively). All other CONSORT items were adequately reported in less than 50% of the abstracts. Notably, we found limited RCT abstracts with adequate reporting on the exact trial design (17%), method of randomization (0%), blinding (0%), number of subjects analyzed (17%), harms (4%), outcomes in both trial arms (13%), as well as the interpretation of the results in the conclusions (13%). Our findings were in agreement with other studies in the medical and dental literature confirming inadequate reporting according to CONSORT guidelines [9, 48–50]. Surprisingly, even after dental journals adhered to the CONSORT for abstracts guidelines [50], those guidelines were not systematically reinforced. Therefore, RCT abstracts were still characterized by inadequate reporting quality.

Consistent with other reports, the CONSORT items most adequately reported in the RCT abstracts were related to objectives and numbers randomized [49, 50]. Journal and article metrics including impact factor or citation metrics were unreliable in predicting reporting quality, as confirmed in other studies [51]. The lack of significance in this correlation could be also related to the small number of included publications.

In the spin analysis, it is noteworthy, that 10 of 24 of the included RCT publications were excluded due to the lack of an explicitly defined single primary outcome. The use of multiple primary outcomes in RCTs combined with the lack of adequate power analysis for multiple outcomes might significantly increase the risk of bias.

The spin analysis according to established criteria [4] showed that various strategies of spin were adapted in the included abstracts (n = 14). Specifically, spin phenomena in either the result or conclusion sections of the abstracts were detected in the majority of the publications. Half of the abstracts presented a tendency for conclusions that had misleading statements designed to exaggerate or falsely claim efficacy. One third of the RCT abstracts presented the trial results in a before-after therapy manner focusing on within group analysis, highlighting statistical significance and ignoring the between group comparisons as directed by the study objectives. More than half of the RCT abstracts emphasized significance in secondary outcomes, a commonly used spin strategy, when the primary outcome results were not significant.

Our results agreed with other studies in the medical literature that investigated spin strategies and misrepresentation of RCT results with various methodologies [4, 5, 51–54]. Austin et al [5] reported some form of spin to exist in 47% of the included RCT abstracts while Cooper et al [53] reported spin to be as high as 70% of the included articles. Interestingly, Pitkin et al [8] compared data reported in the abstract of a random sample of RCTs published in 6 major medical journals to the data presented in the full-text manuscripts and found that inconsistencies at variable levels (18%-68%) existed between data reported in the abstract compared to the full text.

The present study has several strengths. We applied strict inclusion criteria as directed by the research hypothesis and only included RCT publications examining the impact of periodontal intervention on CVD outcomes [55]. We standardized the data extraction methodology utilizing well-defined decision guide and calibration between assessors. Therefore, we have demonstrated a high level of inter-rater agreement with any differences resolved by a third evaluator to ensure greatest accuracy in our analysis. While the present study focused on RCT abstracts alone and not the full text of the included publications, these considerable reporting shortages and/or misrepresentations were a cause for concern, given the wide attention abstracts receive within the healthcare and media communities.

Our study also has some inherent limitations. Although the spin assessment is characterized by subjectivity, two independent and calibrated reviewers per abstract conducted the data extraction and determined the spin strategies. With this method, we aimed to control the magnitude of subjectivity. We employed spin analysis previously used by other groups [1, 4, 6, 53]. Therefore, our analysis was focused on abstract sections and might have missed additional spin strategies present in the full text.

It is important to emphasize that poor reporting quality does not translate into poor study design [56]. It does, however, indicate lack of transparency and prevent the replication of the given experiment [57]. Therefore, quality reporting is necessary for the advancement of science [58]. It is also even more important to emphasize that the identification of spin in an abstract (according to criteria of Boutron et al 2014) should not be taken as a verdict that a research report is fraudulent or fake. The spin strategies examined don’t all carry the same weight in terms of the impact on a reader’s perception of the abstract and it does not assess if these spin strategies were applied in the full text of the manuscript. Nevertheless, low quality reporting and introduction of spin might be contributing to continued controversy in this field of research [59], flawed professional and public perception of research findings [1, 13] and continued ill-advised expenditure of valuable time and resources [60]. The responsibility to improve reporting of RCTs and avoidance of misrepresentations falls on multiple parties. Journal editors and peer reviewers as gatekeepers could reinforce strict practices to ensure adherence to CONSORT or other reporting guidelines, and to require trial registration prior to the commencement of the trial as recommended by the ICMJE [61]. An additional effort by academic institutions, professional organizations, and scientific communities should be exerted to raise awareness among the general scientific audience on proper reporting practices and spin strategies. The scientific community should embrace post-publication appraisal and critique with a goal to improve reporting quality and minimize the incidence of spin.

Conclusions: Poor adherence to the CONSORT for abstracts guidelines and high levels of data “spin” were found in the abstracts of RCTs examining the effect of periodontal therapy on CVD outcomes. Our findings indicate that journal editors and reviewers should reinforce strict adherence to proper reporting guidelines by researchers and article authors to improve reporting quality and reduce spin.

Supporting information

S1 Table. List of the trials register that were used to identify eligible studies.

(DOCX)

Click here for additional data file. (18.9KB, docx)
S1 Fig. Overall Consort score for each article.

Articles were de-identified. Y-axis represents the fulfilled CONSORT items per article.

(TIF)

Click here for additional data file. (92.9KB, tif)
S2 Fig. Overall Spin Score for each article.

The articles were de-identified. Y-axis represents the numbers of fulfilled Spin checklist items.

(TIF)

Click here for additional data file. (97.1KB, tif)
S1 Data

(XLSX)

Click here for additional data file. (18.9KB, xlsx)
S2 Data

(XLSX)

Click here for additional data file. (19.6KB, xlsx)
S3 Data

(XLSX)

Click here for additional data file. (20.3KB, xlsx)

Data Availability

All relevant data are within the paper and its Supporting Information files.

Funding Statement

This manuscript received NIH/NIDDK funding (R21 DK108076) awarded to EI.

References

  • 1.Boutron I, Altman DG, Hopewell S, Vera-Badillo F, Tannock I, Ravaud P. Impact of spin in the abstracts of articles reporting results of randomized controlled trials in the field of cancer: the SPIIN randomized controlled trial. J Clin Oncol. 2014;32(36):4120–6. Epub 2014/11/19. 10.1200/JCO.2014.56.7503 . [DOI] [PubMed] [Google Scholar]
  • 2.Hopewell S, Eisinga A, Clarke M. Better reporting of randomized trials in biomedical journal and conference abstracts. Journal of Information Science. 2008;34((2)):162–73. [Google Scholar]
  • 3.Hopewell S, Clarke M, Moher D, Wager E, Middleton P, Altman DG, et al. CONSORT for reporting randomized controlled trials in journal and conference abstracts: explanation and elaboration. PLoS Med. 2008;5(1):e20 Epub 2008/01/25. 10.1371/journal.pmed.0050020 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Boutron I, Dutton S, Ravaud P, Altman DG. Reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes. JAMA. 2010;303(20):2058–64. Epub 2010/05/27. 10.1001/jama.2010.651 . [DOI] [PubMed] [Google Scholar]
  • 5.Austin J, Smith C, Natarajan K, Som M, Wayant C, Vassar M. Evaluation of spin within abstracts in obesity randomized clinical trials: A cross-sectional review. Clinical obesity. 2019;9(2):e12292 Epub 2018/12/24. 10.1111/cob.12292 . [DOI] [PubMed] [Google Scholar]
  • 6.Lazarus C, Haneef R, Ravaud P, Boutron I. Classification and prevalence of spin in abstracts of non-randomized studies evaluating an intervention. BMC Med Res Methodol. 2015;15:85 Epub 2015/10/16. 10.1186/s12874-015-0079-x [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Li G, Abbade LPF, Nwosu I, Jin Y, Leenus A, Maaz M, et al. A scoping review of comparisons between abstracts and full reports in primary biomedical research. BMC Med Res Methodol. 2017;17(1):181 Epub 2017/12/31. 10.1186/s12874-017-0459-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Pitkin RM, Branagan MA, Burmeister LF. Accuracy of data in abstracts of published research articles. JAMA. 1999;281(12):1110–1. Epub 1999/04/03. 10.1001/jama.281.12.1110 . [DOI] [PubMed] [Google Scholar]
  • 9.Seehra J, Wright NS, Polychronopoulou A, Cobourne MT, Pandis N. Reporting quality of abstracts of randomized controlled trials published in dental specialty journals. J Evid Based Dent Pract. 2013;13(1):1–8. Epub 2013/03/14. 10.1016/j.jebdp.2012.11.001 . [DOI] [PubMed] [Google Scholar]
  • 10.Alharbi F, Almuzian M. The quality of reporting RCT abstracts in four major orthodontics journals for the period 2012–2017. Journal of orthodontics. 2019;46(3):225–34. Epub 2019/07/05. 10.1177/1465312519860160 . [DOI] [PubMed] [Google Scholar]
  • 11.Hua F, Sun Q, Zhao T, Chen X, He H. Reporting quality of randomised controlled trial abstracts presented at the SLEEP Annual Meetings: a cross-sectional study. BMJ open. 2019;9(7):e029270 Epub 2019/07/19. 10.1136/bmjopen-2019-029270 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Roszhart JI, Kumar SS, Allareddy V, Childs CA, Elangovan S. Spin in abstracts of randomized controlled trials in dentistry: A cross-sectional analysis. J Am Dent Assoc. 2019:26–32.e3. Epub 2019/11/22. 10.1016/j.adaj.2019.08.009 . [DOI] [PubMed] [Google Scholar]
  • 13.Yavchitz A, Boutron I, Bafeta A, Marroun I, Charles P, Mantz J, et al. Misrepresentation of randomized controlled trials in press releases and news coverage: a cohort study. PLoS Med. 2012;9(9):e1001308 Epub 2012/09/18. 10.1371/journal.pmed.1001308 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Phillips DP, Kanter EJ, Bednarczyk B, Tastad PL. Importance of the lay press in the transmission of medical knowledge to the scientific community. N Engl J Med. 1991;325(16):1180–3. Epub 1991/10/17. 10.1056/NEJM199110173251620 . [DOI] [PubMed] [Google Scholar]
  • 15.DeStefano F, Anda RF, Kahn HS, Williamson DF, Russell CM. Dental disease and risk of coronary heart disease and mortality. BMJ. 1993;306(6879):688–91. Epub 1993/03/13. 10.1136/bmj.306.6879.688 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Mattila KJ, Nieminen MS, Valtonen VV, Rasi VP, Kesaniemi YA, Syrjala SL, et al. Association between dental health and acute myocardial infarction. BMJ. 1989;298(6676):779–81. Epub 1989/03/25. 10.1136/bmj.298.6676.779 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.American Academy of Periodontology. Periodontal disease linked to cardiovascular disease 2012, April 18 [cited 2018 Nov 6]. Available from: https://perio.org/consumer/AHA-statement.
  • 18.Lockhart PB, Bolger AF, Papapanou PN, Osinbowale O, Trevisan M, Levison ME, et al. Periodontal disease and atherosclerotic vascular disease: does the evidence support an independent association?: a scientific statement from the American Heart Association. Circulation. 2012;125(20):2520–44. Epub 2012/04/20. 10.1161/CIR.0b013e31825719f3 . [DOI] [PubMed] [Google Scholar]
  • 19.International Clinical Trials Registry Platform ICTRP 2018: World Health Organization; [cited 2018]. Available from: https://apps.who.int/trialsearch/.
  • 20.Hujoel PP, Drangsholt M, Spiekerman C, DeRouen TA. Periodontal disease and coronary heart disease risk. Jama. 2000;284(11):1406–10. Epub 2000/09/16. 10.1001/jama.284.11.1406 . [DOI] [PubMed] [Google Scholar]
  • 21.Scopus.com [Internet]. Elsevier; SCOPUS Amsterdam, Natherlands 2018. [cited 12th February 2018]. Available from: https://www.scopus.com/home.uri?zone=header&origin=searchbasic.
  • 22.Clarivate Analytics [Internet]. Web of Science Philadelphia, US: Thomson Reuter; 2018. [cited 25th November 2018]. Available from: https://jcr.clarivate.com. [Google Scholar]
  • 23.Shanthanna H, Kaushal A, Mbuagbaw L, Couban R, Busse J, Thabane L. A cross-sectional study of the reporting quality of pilot or feasibility trials in high-impact anesthesia journals. Canadian journal of anaesthesia = Journal canadien d'anesthesie. 2018;65(11):1180–95. Epub 2018/08/11. 10.1007/s12630-018-1194-z . [DOI] [PubMed] [Google Scholar]
  • 24.Bizzarro S, van der Velden U, Teeuw WJ, Gerdes VEA, Loos BG. Effect of periodontal therapy with systemic antimicrobials on parameters of metabolic syndrome: A randomized clinical trial. J Clin Periodontol. 2017;44(8):833–41. Epub 2017/06/18. 10.1111/jcpe.12763 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Offenbacher S, Beck JD, Moss K, Mendoza L, Paquette DW, Barrow DA, et al. Results from the Periodontitis and Vascular Events (PAVE) Study: a pilot multicentered, randomized, controlled trial to study effects of periodontal therapy in a secondary prevention model of cardiovascular disease. J Periodontol. 2009;80(2):190–201. Epub 2009/02/04. 10.1902/jop.2009.080007 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Bokhari SA, Khan AA, Butt AK, Azhar M, Hanif M, Izhar M, et al. Non-surgical periodontal therapy reduces coronary heart disease risk markers: a randomized controlled trial. J Clin Periodontol. 2012;39(11):1065–74. Epub 2012/09/13. 10.1111/j.1600-051X.2012.01942.x . [DOI] [PubMed] [Google Scholar]
  • 27.Carallo C, Franceschi MS, Tripolino C, Iovane C, Catalano S, Giudice A, et al. Periodontal Treatment Elevates Carotid Wall Shear Stress in the Medium Term. Medicine (Baltimore). 2015;94(42):e1724 Epub 2015/10/27. 10.1097/MD.0000000000001724 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Caula AL, Lira-Junior R, Tinoco EM, Fischer RG. The effect of periodontal therapy on cardiovascular risk markers: a 6-month randomized clinical trial. J Clin Periodontol. 2014;41(9):875–82. Epub 2014/07/22. 10.1111/jcpe.12290 . [DOI] [PubMed] [Google Scholar]
  • 29.Cortelli SC, Costa FO, Rodrigues E, Cota LO, Cortelli JR. Periodontal Therapy Effects on Nitrite Related to Oral Bacteria: A 6-Month Randomized Clinical Trial. J Periodontol. 2015;86(8):984–94. Epub 2015/03/27. 10.1902/jop.2015.140678 . [DOI] [PubMed] [Google Scholar]
  • 30.D'Aiuto F, Nibali L, Parkar M, Suvan J, Tonetti MS. Short-term effects of intensive periodontal therapy on serum inflammatory markers and cholesterol. J Dent Res. 2005;84(3):269–73. Epub 2005/02/23. 10.1177/154405910508400312 . [DOI] [PubMed] [Google Scholar]
  • 31.D'Aiuto F, Parkar M, Nibali L, Suvan J, Lessem J, Tonetti MS. Periodontal infections cause changes in traditional and novel cardiovascular risk factors: results from a randomized controlled clinical trial. Am Heart J. 2006;151(5):977–84. Epub 2006/04/29. 10.1016/j.ahj.2005.06.018 . [DOI] [PubMed] [Google Scholar]
  • 32.Fu YW, Li XX, Xu HZ, Gong YQ, Yang Y. Effects of periodontal therapy on serum lipid profile and proinflammatory cytokines in patients with hyperlipidemia: a randomized controlled trial. Clin Oral Investig. 2016;20(6):1263–9. Epub 2015/10/06. 10.1007/s00784-015-1621-2 . [DOI] [PubMed] [Google Scholar]
  • 33.Oz SG, Fentoglu O, Kilicarslan A, Guven GS, Tanrtover MD, Aykac Y, et al. Beneficial effects of periodontal treatment on metabolic control of hypercholesterolemia. South Med J. 2007;100(7):686–91. Epub 2007/07/21. 10.1097/SMJ.0b013e31802fa327 . [DOI] [PubMed] [Google Scholar]
  • 34.Gunupati S, Chava VK, Krishna BP. Effect of phase I periodontal therapy on anti-cardiolipin antibodies in patients with acute myocardial infarction associated with chronic periodontitis. J Periodontol. 2011;82(12):1657–64. Epub 2011/04/14. 10.1902/jop.2011.110002 . [DOI] [PubMed] [Google Scholar]
  • 35.Hada DS, Garg S, Ramteke GB, Ratre MS. Effect of Non-Surgical Periodontal Treatment on Clinical and Biochemical Risk Markers of Cardiovascular Disease: A Randomized Trial. J Periodontol. 2015;86(11):1201–11. Epub 2015/07/25. 10.1902/jop.2015.150249 . [DOI] [PubMed] [Google Scholar]
  • 36.Ide M, McPartlin D, Coward PY, Crook M, Lumb P, Wilson RF. Effect of treatment of chronic periodontitis on levels of serum markers of acute-phase inflammatory and vascular responses. J Clin Periodontol. 2003;30(4):334–40. Epub 2003/04/16. 10.1034/j.1600-051x.2003.00282.x . [DOI] [PubMed] [Google Scholar]
  • 37.Javed F, Kellesarian SV, Al-Kheraif AA, Ranna V, Qadri T, Yunker M, et al. Effect of Nd:YAG laser-assisted non-surgical periodontal therapy on clinical periodontal and serum biomarkers in patients with and without coronary artery disease: A short-term pilot study. Lasers Surg Med. 2016;48(10):929–35. Epub 2016/02/06. 10.1002/lsm.22483 . [DOI] [PubMed] [Google Scholar]
  • 38.Kamil W, Al Habashneh R, Khader Y, Al Bayati L, Taani D. Effects of nonsurgical periodontal therapy on C-reactive protein and serum lipids in Jordanian adults with advanced periodontitis. J Periodontal Res. 2011;46(5):616–21. Epub 2011/06/03. 10.1111/j.1600-0765.2011.01380.x . [DOI] [PubMed] [Google Scholar]
  • 39.Kapellas K, Maple-Brown LJ, Jamieson LM, Do LG, O'Dea K, Brown A, et al. Effect of periodontal therapy on arterial structure and function among aboriginal australians: a randomized, controlled trial. Hypertension. 2014;64(4):702–8. Epub 2014/06/25. 10.1161/HYPERTENSIONAHA.114.03359 . [DOI] [PubMed] [Google Scholar]
  • 40.Li X, Tse HF, Yiu KH, Li LS, Jin L. Effect of periodontal treatment on circulating CD34(+) cells and peripheral vascular endothelial function: a randomized controlled trial. J Clin Periodontol. 2011;38(2):148–56. Epub 2010/12/08. 10.1111/j.1600-051X.2010.01651.x . [DOI] [PubMed] [Google Scholar]
  • 41.Lopez NJ, Quintero A, Casanova PA, Ibieta CI, Baelum V, Lopez R. Effects of periodontal therapy on systemic markers of inflammation in patients with metabolic syndrome: a controlled clinical trial. J Periodontol. 2012;83(3):267–78. Epub 2011/07/14. 10.1902/jop.2011.110227 . [DOI] [PubMed] [Google Scholar]
  • 42.Taylor B, Tofler G, Morel-Kopp MC, Carey H, Carter T, Elliott M, et al. The effect of initial treatment of periodontitis on systemic markers of inflammation and cardiovascular risk: a randomized controlled trial. Eur J Oral Sci. 2010;118(4):350–6. Epub 2010/07/29. 10.1111/j.1600-0722.2010.00748.x . [DOI] [PubMed] [Google Scholar]
  • 43.Tonetti MS, D'Aiuto F, Nibali L, Donald A, Storry C, Parkar M, et al. Treatment of periodontitis and endothelial function. N Engl J Med. 2007;356(9):911–20. Epub 2007/03/03. 10.1056/NEJMoa063186 . [DOI] [PubMed] [Google Scholar]
  • 44.Tuter G, Kurtis B, Serdar M, Aykan T, Okyay K, Yucel A, et al. Effects of scaling and root planing and sub-antimicrobial dose doxycycline on oral and systemic biomarkers of disease in patients with both chronic periodontitis and coronary artery disease. J Clin Periodontol. 2007;34(8):673–81. Epub 2007/06/26. 10.1111/j.1600-051X.2007.01104.x . [DOI] [PubMed] [Google Scholar]
  • 45.Ushida Y, Koshy G, Kawashima Y, Kiji M, Umeda M, Nitta H, et al. Changes in serum interleukin-6, C-reactive protein and thrombomodulin levels under periodontal ultrasonic debridement. J Clin Periodontol. 2008;35(11):969–75. Epub 2008/11/04. 10.1111/j.1600-051X.2008.01316.x . [DOI] [PubMed] [Google Scholar]
  • 46.Vidal F, Figueredo CM, Cordovil I, Fischer RG. Periodontal therapy reduces plasma levels of interleukin-6, C-reactive protein, and fibrinogen in patients with severe periodontitis and refractory arterial hypertension. J Periodontol. 2009;80(5):786–91. Epub 2009/05/02. 10.1902/jop.2009.080471 . [DOI] [PubMed] [Google Scholar]
  • 47.Zhou QB, Xia WH, Ren J, Yu BB, Tong XZ, Chen YB, et al. Effect of Intensive Periodontal Therapy on Blood Pressure and Endothelial Microparticles in Patients With Prehypertension and Periodontitis: A Randomized Controlled Trial. J Periodontol. 2017;88(8):711–22. Epub 2017/04/30. 10.1902/jop.2017.160447 . [DOI] [PubMed] [Google Scholar]
  • 48.Faggion CM Jr., Giannakopoulos NN. Quality of reporting in abstracts of randomized controlled trials published in leading journals of periodontology and implant dentistry: a survey. J Periodontol. 2012;83(10):1251–6. Epub 2012/01/10. 10.1902/jop.2012.110609 . [DOI] [PubMed] [Google Scholar]
  • 49.Fleming PS, Buckley N, Seehra J, Polychronopoulou A, Pandis N. Reporting quality of abstracts of randomized controlled trials published in leading orthodontic journals from 2006 to 2011. Am J Orthod Dentofacial Orthop. 2012;142(4):451–8. Epub 2012/09/25. 10.1016/j.ajodo.2012.05.013 . [DOI] [PubMed] [Google Scholar]
  • 50.Hua F, Deng L, Kau CH, Jiang H, He H, Walsh T. Reporting quality of randomized controlled trial abstracts: survey of leading general dental journals. J Am Dent Assoc. 2015;146(9):669–78 e1. Epub 2015/09/01. 10.1016/j.adaj.2015.03.020 . [DOI] [PubMed] [Google Scholar]
  • 51.Patel SV, Chadi SA, Choi J, Colquhoun PH. The use of "spin" in laparoscopic lower GI surgical trials with nonsignificant results: an assessment of reporting and interpretation of the primary outcomes. Diseases of the colon and rectum. 2013;56(12):1388–94. Epub 2013/11/10. 10.1097/01.dcr.0000436466.50341.c5 . [DOI] [PubMed] [Google Scholar]
  • 52.Amos AJ. A review of spin and bias use in the early intervention in psychosis literature. The primary care companion for CNS disorders. 2014;16(1). Epub 2014/06/19. 10.4088/PCC.13r01586 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53.Cooper CM, Gray HM, Ross AE, Hamilton TA, Bea Downs J, Wayant C, et al. Evaluation of spin in the abstracts of otolaryngology randomized controlled trials. The Laryngoscope. 2018. Epub 2018/12/24. 10.1002/lary.27750 . [DOI] [PubMed] [Google Scholar]
  • 54.Vera-Badillo FE, Shapiro R, Ocana A, Amir E, Tannock IF. Bias in reporting of end points of efficacy and toxicity in randomized, clinical trials for women with breast cancer. Ann Oncol. 2013;24(5):1238–44. Epub 2013/01/11. 10.1093/annonc/mds636 . [DOI] [PubMed] [Google Scholar]
  • 55.Li C, Lv Z, Shi Z, Zhu Y, Wu Y, Li L, et al. Periodontal therapy for the management of cardiovascular disease in patients with chronic periodontitis. The Cochrane database of systematic reviews. 2017;11:Cd009197 Epub 2017/11/08. 10.1002/14651858.CD009197.pub3 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 56.Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA. 1995;273(5):408–12. Epub 1995/02/01. 10.1001/jama.273.5.408 . [DOI] [PubMed] [Google Scholar]
  • 57.Simera I, Moher D, Hirst A, Hoey J, Schulz KF, Altman DG. Transparent and accurate reporting increases reliability, utility, and impact of your research: reporting guidelines and the EQUATOR Network. BMC Medicine. 2010;8(1):24 10.1186/1741-7015-8-24 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 58.Moher D. Reporting guidelines: doing better for readers. BMC Med. 2018;16(1):233 Epub 2018/12/14. 10.1186/s12916-018-1226-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 59.Pihlstrom BL, Hodges JS, Michalowicz B, Wohlfahrt JC, Garcia RI. Promoting oral health care because of its possible effect on systemic disease is premature and may be misleading. J Am Dent Assoc. 2018;149(6):401–3. Epub 2018/05/29. 10.1016/j.adaj.2018.03.030 . [DOI] [PubMed] [Google Scholar]
  • 60.Ioannidis JP, Greenland S, Hlatky MA, Khoury MJ, Macleod MR, Moher D, et al. Increasing value and reducing waste in research design, conduct, and analysis. Lancet (London, England). 2014;383(9912):166–75. Epub 2014/01/15. 10.1016/s0140-6736(13)62227-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 61.International Committee of Medical Journal Editors. Recommendations of the conduct, reporting, editing, and publication of scholarly work in medical journals 2018 [April 15th, 2019]. Available from: http://www.icmje.org/recommendations/browse/. [PubMed]

Decision Letter 0

Dermot Cox

9 Jan 2020

PONE-D-19-31524

Reporting quality and spin in abstracts of randomized clinical trials of periodontal therapy and cardiovascular disease outcomes

PLOS ONE

Dear Dr Ioannidou,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Both reviewers found this to be an interesting paper and worthy of publication. However, they did have a few issues that they would like addressed, in particular, the relationship between spin and propaganda.

We would appreciate receiving your revised manuscript by Feb 23 2020 11:59PM. When you are ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter.

To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

Please include the following items when submitting your revised manuscript:

  • A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). This letter should be uploaded as separate file and labeled 'Response to Reviewers'.

  • A marked-up copy of your manuscript that highlights changes made to the original version. This file should be uploaded as separate file and labeled 'Revised Manuscript with Track Changes'.

  • An unmarked version of your revised paper without tracked changes. This file should be uploaded as separate file and labeled 'Manuscript'.

Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.

We look forward to receiving your revised manuscript.

Kind regards,

Dermot Cox

Academic Editor

PLOS ONE

Journal requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

http://www.journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and http://www.journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please upload a copy of Figure 1, to which you refer in your text on page 7 and 10. If the figure is no longer to be included as part of the submission please remove all reference to it within the text.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: N/A

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: This is an interesting topic, and one that is gathering pace recently – probably belatedly! I have very few comments, mainly to do with phrasing and updating the literature.

1. Introduction

1.1 Page 4: Spin is probably not the same as propaganda, since spin is the biased reporting of factual information in such a way as to mislead the reader, while propaganda can (and usually does) include lies.

1.2 The introduction is well written, but needs a little updating to reflect recent publications. The authors might want to check these :

1. Alharbi F, Almuzian M. The quality of reporting RCT abstracts in four major orthodontics journals for the period 2012-2017. J Orthod. 2019;46(3):225-34.

2. Austin J, Smith C, Natarajan K, Som M, Wayant C, Vassar M. Evaluation of spin within abstracts in obesity randomized clinical trials: A cross-sectional review. Clin Obes. 2019;9(2):e12292.

3. Boutron I, Haneef R, Yavchitz A, Baron G, Novack J, Oransky I, et al. Three randomized controlled trials evaluating the impact of "spin" in health news stories reporting studies of pharmacologic treatments on patients'/caregivers' interpretation of treatment benefit. BMC Med. 2019;17(1):105.

4. Hua F, Sun Q, Zhao T, Chen X, He H. Reporting quality of randomised controlled trial abstracts presented at the SLEEP Annual Meetings: a cross-sectional study. BMJ Open. 2019;9(7):e029270.

5. Roszhart JI, Kumar SS, Allareddy V, Childs CA, Elangovan S. Spin in abstracts of randomized controlled trials in dentistry: A cross-sectional analysis. J Am Dent Assoc. 2019.

2. Methods

Clearly described.

2.1 Page 10 : Cohen's kappa is a statistic, not a test.

2.2 Page 10 : 'non parametric' should be omitted. Statistical procedures are parametric or nonparametric, but data are just data.

3. Results

3.1 The results are interesting, but of course with a small sample they come with wide confidence intervals. I would include these in table 4, as these constitute the focal findings of the study. For instance, claiming efficacy with no consideration of non-significant primary outcome, which is 9 of 14 studies is 64% with a CI of 39% to 84%. So even if the lower figure were the true prevalence, it would still be pretty damning. I don't know if SPSS can be persuaded to do reasonable confidence intervals for small sample sizes (I haven't used it for some years) but the authors should use either Wilson's or Jeffrey's confidence interval. See

Brown, L., Cai, T., DasGupta, A. (2001). Interval estimation for a binomial proportion Statistical Science 16(2), 101 - 117. https://dx.doi.org/10.1214/ss/1009213286

for a discussion of the various formulas for confidence intervals.

3.2 I'd like an explanation of "inadequate extrapolation to larger population" – does this mean overgeneralising the results, or failing to provide adequate confidence intervals? Both, I think, are sins.

3.3 Page 20 : "made hyped statements" hype is deception carried out in publicity. But it can also mean exaggerated claims. I think that "misleading statements designed to exaggerate or falsely claim efficacy" is clearer, and makes explicit what is going in.

3.4 The abbreviations for reporting quality score and spin score make the paper less readable, and reduce the impact. Abbreviations don't save paper, so the authors should confine themselves to ones that are used in everyday language (eg everyone says RCT nowadays) or ones that are accepted acronyms (eg CONSORT)

3.5 The graphs are not helpful because they display the data in no sort of order, leaving the reader unable to form an overall impression. I suggest boxplots with data shown, as in the uploaded picture (produced by jamovi, a free, open-source package that is going to eat SPSS!)

4. Discussion

4.1 "deemed to be poor" = poor.

4.2 "in regard to the spin analysis" = "in the analysis of spin"

4.3 I would prefer "intentionally misleading conclusions" to "hyped conclusions"

Reviewer #2: The take of spin is of course exiting and novel, at least in the dental field. The authors start off with declaring that Spin is just another word for propaganda and then present a list of 20 points that need to be included in an abstract to not be considered as propaganda. In the age of fake news this can easily be misinterpreted as all research paper abstracts failing one of the points should be considered to be propaganda.

The results section needs to be more focused on Spin and its criteria. The abstract is more spin oriented than the results section, which is a bit ironic.

In the discussion section the authors need to add a section in which they discuss the concept of spin – to use the same verdict for insinuating that non-significant data are significant as for putting some spotlight on a secondary outcome that turned out to be interesting than the authors thought when designing the study is something that should be discussed. It needs to be made extremely clear (also in the abstract) that an abstract failing one of the 20 point (in table 4) is not the same thing as fraudulent research. The paper and the research its presenting could still be excellent.

Finally the authors need to compare their data to similar Spin analysis in other medical fields. Right now that’s just one sentence, it need to be elaborated.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Ronán M Conroy, DSc

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files to be viewed.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org. Please note that Supporting Information files do not need this step.

PLoS One. 2020 Apr 17;15(4):e0230843. doi: 10.1371/journal.pone.0230843.r002

Author response to Decision Letter 0

18 Feb 2020

Reviewer #1: This is an interesting topic, and one that is gathering pace recently – probably belatedly! I have very few comments, mainly to do with phrasing and updating the literature.

We thank the review for the kind words.

1. Introduction

1.1 Page 4: Spin is probably not the same as propaganda, since spin is the biased reporting of factual information in such a way as to mislead the reader, while propaganda can (and usually does) include lies.

The reviewer raises a valid concern. Therefore, the term ”propaganda” was removed from the text.

1.2 The introduction is well written, but needs a little updating to reflect recent publications. The authors might want to check these:

1. Alharbi F, Almuzian M. The quality of reporting RCT abstracts in four major orthodontics journals for the period 2012-2017. J Orthod. 2019;46(3):225-34.

2. Austin J, Smith C, Natarajan K, Som M, Wayant C, Vassar M. Evaluation of spin within abstracts in obesity randomized clinical trials: A cross-sectional review. Clin Obes. 2019;9(2):e12292.

3. Boutron I, Haneef R, Yavchitz A, Baron G, Novack J, Oransky I, et al. Three randomized controlled trials evaluating the impact of "spin" in health news stories reporting studies of pharmacologic treatments on patients'/caregivers' interpretation of treatment benefit. BMC Med. 2019;17(1):105.

4. Hua F, Sun Q, Zhao T, Chen X, He H. Reporting quality of randomised controlled trial abstracts presented at the SLEEP Annual Meetings: a cross-sectional study. BMJ Open. 2019;9(7):e029270.

5. Roszhart JI, Kumar SS, Allareddy V, Childs CA, Elangovan S. Spin in abstracts of randomized controlled trials in dentistry: A cross-sectional analysis. J Am Dent Assoc. 2019.

We appreciate highlighting these references to us. The papers as recommended by the reviewer were cited appropriately.

2. Methods

Clearly described.

2.1 Page 10: Cohen's kappa is a statistic, not a test.

We revised the manuscript accordingly and the word “test” was changed into “statistic”.

2.2 Page 10: 'non parametric' should be omitted. Statistical procedures are parametric or nonparametric, but data are just data.

The phrase “non parametric” was removed in page 10.

3. Results

3.1 The results are interesting, but of course with a small sample they come with wide confidence intervals. I would include these in table 4, as these constitute the focal findings of the study. For instance, claiming efficacy with no consideration of non-significant primary outcome, which is 9 of 14 studies is 64% with a CI of 39% to 84%. So even if the lower figure were the true prevalence, it would still be pretty damning. I don't know if SPSS can be persuaded to do reasonable confidence intervals for small sample sizes (I haven't used it for some years) but the authors should use either Wilson's or Jeffrey's confidence interval. See

Brown, L., Cai, T., DasGupta, A. (2001). Interval estimation for a binomial proportion Statistical Science 16(2), 101 - 117. https://dx.doi.org/10.1214/ss/1009213286

for a discussion of the various formulas for confidence intervals.

Although the reviewer raises a good point, the limited sample size prevented us from using inferential statistics to avoid misleading conclusions.

3.2 I'd like an explanation of "inadequate extrapolation to larger population" – does this mean over-generalizing the results, or failing to provide adequate confidence intervals? Both, I think, are sins.

Boutron et al. define this item as “over generalizing the results”. Therefore, when we applied the checklist, we followed their definition.

3.3 Page 20: "made hyped statements" hype is deception carried out in publicity. But it can also mean exaggerated claims. I think that "misleading statements designed to exaggerate or falsely claim efficacy" is clearer, and makes explicit what is going in.

We agreed with the recommendation. Therefore, the item title was changed into “misleading statements designed to exaggerate or falsely claim efficacy”.

3.4 The abbreviations for reporting quality score and spin score make the paper less readable, and reduce the impact. Abbreviations don't save paper, so the authors should confine themselves to ones that are used in everyday language (e.g. everyone says RCT nowadays) or ones that are accepted acronyms (e.g. CONSORT)

We agreed with the recommendation and we revised the manuscript accordingly.

3.5 The graphs are not helpful because they display the data in no sort of order, leaving the reader unable to form an overall impression. I suggest boxplots with data shown, as in the uploaded picture (produced by jamovi, a free, open-source package that is going to eat SPSS!)

We agreed with the recommendation and we revised the supplement figures into box plots.

4. Discussion

4.1 "deemed to be poor" = poor.

4.2 "in regard to the spin analysis" = "in the analysis of spin"

4.3 I would prefer "intentionally misleading conclusions" to "hyped conclusions"

We agreed with the recommendation and these rephrasing suggestions were adopted.

Reviewer #2:

Comment 1: The take of spin is of course exiting and novel, at least in the dental field. The authors start off with declaring that Spin is just another word for propaganda and then present a list of 20 points that need to be included in an abstract to not be considered as propaganda. In the age of fake news this can easily be misinterpreted as all research paper abstracts failing one of the points should be considered to be propaganda.

The reviewer raises a valid concern. We strived to be sensitive to this aspect both during the preparation of the manuscript and during this review process. Accordingly, and to avoid sending out a wrong message, the term “propaganda” was removed. Also, some changes in the wording of the abstract were made to reflect our findings, as much as an abstract allows, with least possible misinterpretation. Moreover, the distinction between poor reporting and poor research design/ execution, and between presence of spin and fraud was highlighted and emphasized further in our discussion section.

Comment 2: The results section needs to be more focused on Spin and its criteria. The abstract is more spin oriented than the results section, which is a bit ironic.

The criteria of spin according to Boutron et al are outlined/elaborated in table 4. In addition, our findings regarding each spin criterion were presented. Our reporting of results highlights these findings. We thank the reviewer for the comment and we made an effort to balance CONSORT and Spin in in both abstract and result sections.

Comment 3: In the discussion section the authors need to add a section in which they discuss the concept of spin – to use the same verdict for insinuating that non-significant data are significant as for putting some spotlight on a secondary outcome that turned out to be interesting than the authors thought when designing the study is something that should be discussed. It needs to be made extremely clear (also in the abstract) that an abstract failing one of the 20 point (in table 4) is not the same thing as fraudulent research. The paper and the research its presenting could still be excellent.

The reviewer raises a valid and concern and we refer him to our response to his first comment/ recommendation. Specifically, changes were made in the abstract and we included a dedicated paragraph in the discussion section to highlight this point

Comment 4: Finally, the authors need to compare their data to similar Spin analysis in other medical fields. Right now, that’s just one sentence, it needs to be elaborated.

We agree with the recommendation and we elaborated on our discussion of relevant medical literature. We emphasized similarities to our findings but also an interesting finding of discordance between abstracts and the full text of a sample of studied RCT’s.

In summary, we hope that the revisions are satisfactory and the manuscript is now accepted for publication.

Sincerely

Effie Ioannidou

UCONN Health

Ioannidou@uchc.edu

Attachment

Submitted filename: Response to Reviewers.docx

Click here for additional data file. (21.9KB, docx)

Decision Letter 1

Dermot Cox

11 Mar 2020

Reporting quality and spin in abstracts of randomized clinical trials of periodontal therapy and cardiovascular disease outcomes

PONE-D-19-31524R1

Dear Dr. Ioannidou,

We are pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it complies with all outstanding technical requirements.

Within one week, you will receive an e-mail containing information on the amendments required prior to publication. When all required modifications have been addressed, you will receive a formal acceptance letter and your manuscript will proceed to our production department and be scheduled for publication.

Shortly after the formal acceptance letter is sent, an invoice for payment will follow. To ensure an efficient production and billing process, please log into Editorial Manager at https://www.editorialmanager.com/pone/, click the "Update My Information" link at the top of the page, and update your user information. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to enable them to help maximize its impact. If they will be preparing press materials for this manuscript, you must inform our press team as soon as possible and no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

With kind regards,

Dermot Cox

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The authors have made a good job of the revision. I tend to agree with them that for small numbers like this, and given the lack of clarity around what population could be generalised to, confidence intervals are not appropriate.

Reviewer #2: Well done! Accepted as it stands. The paper has actually made an impression on me and I'm looking forward to citing it.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Ronán M Conroy

Reviewer #2: No

Acceptance letter

Dermot Cox

23 Mar 2020

PONE-D-19-31524R1

Reporting quality and spin in abstracts of randomized clinical trials of periodontal therapy and cardiovascular disease outcomes

Dear Dr. Ioannidou:

I am pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please notify them about your upcoming paper at this point, to enable them to help maximize its impact. If they will be preparing press materials for this manuscript, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

For any other questions or concerns, please email plosone@plos.org.

Thank you for submitting your work to PLOS ONE.

With kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Dermot Cox

Academic Editor

PLOS ONE

Associated Data

    This section collects any data citations, data availability statements, or supplementary materials included in this article.

    Supplementary Materials

    S1 Table. List of the trials register that were used to identify eligible studies.

    (DOCX)

    Click here for additional data file. (18.9KB, docx)
    S1 Fig. Overall Consort score for each article.

    Articles were de-identified. Y-axis represents the fulfilled CONSORT items per article.

    (TIF)

    Click here for additional data file. (92.9KB, tif)
    S2 Fig. Overall Spin Score for each article.

    The articles were de-identified. Y-axis represents the numbers of fulfilled Spin checklist items.

    (TIF)

    Click here for additional data file. (97.1KB, tif)
    S1 Data

    (XLSX)

    Click here for additional data file. (18.9KB, xlsx)
    S2 Data

    (XLSX)

    Click here for additional data file. (19.6KB, xlsx)
    S3 Data

    (XLSX)

    Click here for additional data file. (20.3KB, xlsx)
    Attachment

    Submitted filename: Response to Reviewers.docx

    Click here for additional data file. (21.9KB, docx)

    Data Availability Statement

    All relevant data are within the paper and its Supporting Information files.

    Articles from PLoS ONE are provided here courtesy of PLOS

    RESOURCES