Abstract
INTRODUCTION
Abstracts presented at national and international scientific meetings are an important educational resource. However, the work is not peer reviewed and little is known about the quality or validity of the presented results and the fate of such abstracts.
MATERIALS AND METHODS
This is a retrospective cohort study of abstracts presented to the 1997 annual meeting of the Association of Surgeons of Great Britain and Ireland. We examined the rates of full-text publication, time to publication, factors influencing publication, inconsistencies between presented and subsequently published manuscripts, and reasons for non-publication of abstracts.
RESULTS
Of the 241 abstracts presented, 136 (56.4%) were published at a median duration of 18 months. Multicentre studies had a greater tendency to subsequent publication and studies involving academic centres predicted publication in a high impact factor journal. Inconsistencies between presented and published abstracts were common and were significantly associated with delayed publication. Oral and poster presentations were equally likely to be published. Reasons for non-submission of presented abstracts included lack of time, low priority to publish, perceived methodological limitations, lack of novelty of findings and co-investigators leaving the organisation.
CONCLUSIONS
More than half of the work presented at a national surgical meeting in the UK has been subsequently published. Various factors that influence the process of publication and remediable causes for non-publication have been identified.
Keywords: Abstracts, Peer-reviewed publication
Meetings organised by national surgical associations are key forums for the communication of clinical/research findings. Such meetings generate testable hypotheses, stimulate debate, facilitate development of guidelines and encourage consensus in healthcare. Little research has, however, been done to assess the quality of abstracts and their validity and usefulness. Abstracts may not contain enough information necessary for readers to assess their validity.1,2 Lack of peer review of the full content could influence quality and consumer confidence. Abstract quality is thought to be associated with the likelihood of subsequent publication of a complete paper in a peer-reviewed journal.3 Furthermore, there is wide-spread belief that such abstracts could be taken as informative, valid and accessible only if they have passed through the rigors of peer review and subsequent full-text publication in appropriate journals. Rates of publication of complete articles after presentation of abstracts at medical/scientific international meetings have ranged from 11% to 78%.3 We aimed to study the fate of abstracts presented to a UK national surgical meeting and assess rates of subsequent publication, time taken to publish, factors influencing publication, consistency of reporting between the presented abstracts and the published abstracts, and reasons for non-publication.
Materials and Methods
Abstracts presented at the 1997 annual meeting of the Association of Surgeons of Great Britain and Ireland [Br J Surg 1997; 84 Suppl 1: 1–71] were evaluated for the following: type of presentation (oral or poster); speciality; number of authors; number and type of organisations; nature of work; study design; statistical methods; sample sizes; and numerical results. To determine full-text publication, PubMed and EMBASE databases were searched (from 1997 to July 2003) by two observers (SPB and IDK) using combinations of author names and key words. Five additional publications were picked up from the respondents to the questionnaire survey. The impact factor of the journals for the year in which the paper was published was retrieved from Journal Citation Reports. Inconsistencies between presented and published abstracts (classified as minor and major) were evaluated by SPB and IDS. ‘Minor inconsistency’ was considered to be present if one or more of the following was found: change in the meaning or interpretation of the study title; change in the number of authors or the authors themselves; change in the sample sizes; obvious change in the methods of statistical analyses; minor changes in study results or precision measures; and change in interpretation of results. ‘Major inconsistency’ was considered to be present if one or more of the following was found: changes in study objective and/or hypothesis; change in study design; and major change in study results or precision measures. The two evaluators assessed inconsistencies independently and then reached a consensus. For abstracts that were considered ‘unpublished’ on the database searches, the main (last or the first) authors were surveyed by a postal questionnaire to determine the following: confirmation of non-publication (or publication in alternative format); attempts at submission for full-text publication; and, if not submitted, reasons for non-submission. A reminder was sent at 6 weeks to all non-responders.
Statistical analysis
Data were analysed using SPSS for Windows (v. 11.5). Non-parametric data were analysed using the Chi square test with Yates correction. Multivariate analyses were done using logistic regression for binary dependent variables and Cox proportional hazards modelling for ‘time to publication’ analyses.
Results
The number of abstracts presented at the annual meeting of the ASGBI in 1997 was 241 (142 oral presentations and 99 posters). Table 1 shows the characteristics of the abstracts and it is evident that there was no difference in the abstract characteristics and quality measures such as use of statistical methods, mention of sample sizes and numerical results between the oral and poster forms of presentations.
Table 1.
Frequencies of abstracts in the various subcategories
| Abstract characteristics | Oral (%) | Poster (%) | P-value* | |
|---|---|---|---|---|
| Speciality | Vascular & transplant | 48 (59.3) | 33 (40.7) | 0.945 |
| General | 23 (54.8) | 19 (45.2) | ||
| LGI | 22 (56.4) | 17 (43.6) | ||
| UGI & H&P | 32 (62.7) | 19 (37.3) | ||
| Breast & endocrine | 17 (60.7) | 11 (39.3) | ||
| Field of work | Clinical | 129 (60.6%) | 84 (39.4%) | 0.30 |
| Translational | 13 (48.1%) | 14 (51.9%) | ||
| Study design | Observational | 87 (54.4) | 73 (45.6) | 0.13 |
| Interventional | 37 (67.3) | 18 (32.7) | ||
| Instrument/diagnostic test validation | 18 (69.2) | 8 (30.8) | ||
| Animal Study | Yes | 3 (37.5) | 5 (62.5) | 0.37 |
| No | 139 (59.7) | 94 (40.3) | ||
| Country | UK and Ireland | 139 (59.4) | 95 (40.6) | 0.63 |
| Elsewhere | 3 (42.9) | 4 (57.1) | ||
| Academic/university/medical school/research institute | Yes | 67 (57.3) | 50 (42.7) | 0.71 |
| No | 72 (60.5) | 47 (39.5) | ||
| Multicentre study | Yes | 23 (54.8) | 19 (45.2) | 0.67 |
| No | 119 (59.8) | 80 (40.2) | ||
| Statistical methods mentioned | Yes | 48 (53.3) | 42 (46.7) | 0.22 |
| No | 94 (62.3) | 57 (37.7) | ||
| Sample sizes mentioned | Yes | 135 (59.2) | 93 (40.8) | 0.74 |
| No | 6 (50.0) | 6 (50.0) | ||
| Numerical results mentioned | Yes | 132 (59.7) | 89 (40.3) | 0.42 |
| No | 9 (47.4) | 10 (52.6) |
Chi-squared test with Yates correction.
Of the 241 abstracts, 136 (56.4%) were subsequently published as a complete report. Publication dates for the full-text articles ranged from the year of presentation (1997) to 2003 (Fig. 1). The median time (inter-quartile range) to publication was 18 months (11, 27). The median (inter-quartile range) impact factor of the published journals (ascertained for 120 publications) was 2 (1.07, 2.38).
Figure 1.
Year of publication of the 136 abstracts.
Multivariate logistic regression analysis including factors such as involvement of academic institution, multicentre or single centre, clinical or translational, study design (observational, interventional and others), speciality, presentation type (oral or poster), mention of statistical methods, sample size, numerical results and author number showed that none of the factors significantly influenced publication, although multicentre studies were more likely to be published than single centre ones with an odds ratio of 2.1 (95% CI were 0.96 and 4.57; P = 0.06). Similar logistic regression analyses showed that abstracts involving academic centres were found to influence publication in a high impact factor journal (defined as more than 2) when compared to non-academic ones with an odds ratio of 2.8 (95% CI were 1.13 and 6.96; P = 0.027). Examining the influence of the above mentioned factors on the time to subsequent publication, a Cox proportional hazards regression model showed that studies involving academic centres were associated with longer times to publish (P = 0.034).
The presented and published abstracts were compared with regard to changes in authors, title of the study, study objective/hypothesis, study design, sample sizes, statistical methods used, results, interpretation and if any additional work was done (Table 2). As only the abstracts of both the presented and published reports were evaluated for inconsistency, the inconsistency rate is likely to be under-estimated. To determine variables associated with such inconsistency, the following factors were analysed using logistic regression: involvement of academic centres; multicentre study; author numbers in the presented abstract; presentation type; study design; and time to publication. It was found that only time to publication was associated significantly with the presence of either minor or major inconsistency between the presented and published abstract (P = 0.008).
Table 2.
Consistency between presented and published abstracts
| Characteristics assessed in the pairs of abstracts | Number/total assesseh | Percentage | 95% confidence interval | |
|---|---|---|---|---|
| Additional work done | 17/127 | 13.4 | 8.5–20.4 | |
| Significant change in study title | 12/136 | 8.8 | 5.1–14.8 | |
| Changes in authorship | Increase | 41/135 | 30.4 | 23.3–38.6 |
| Decrease | 32/135 | 23.7 | 17.3–31.5 | |
| Changea | 6/135 | 4.4 | 2–9.4 | |
| Changes in sample size | Yes – larger | 55/116 | 47.4 | 38.6–56.4 |
| Yes – smaller | 10/116 | 8.6 | 4.8–15.1 | |
| Statistical methods | Mentioned – as before | 27/128 | 21.1 | 14.9–29 |
| Mentioned – changed | 5/128 | 3.9 | 1.7–8.8 | |
| Changes in results | Yes – minor | 74/117 | 63.2 | 54.2–71.4 |
| Yes – major | 4/117 | 3.4 | 1.3–8.5 | |
| Changes in results' interpretation | 15/129 | 11.6 | 7.2–18.3 | |
| Changes in study objective/hypothesis | 2/128 | 1.6 | 0.4–5.5 | |
| Changes in study design | 3/128 | 2.3 | 0.8–6.7 | |
| Minor inconsistenciesb | 110/128 | 85.9 | 78.9–90.9 | |
| Major inconsistenciesb | 7/123 | 5.7 | 2.8–11.3 |
Implies change in authors without change in numbers.
As defined in Materials and Methods.
The authors of 85 abstracts initially assessed as not published were surveyed – 49 (57.6%) responded. Of these, 11 abstracts had been published either as full-text manuscript in journals (8), or in alternative media including theses and books (4) and one had been accepted for publication at the time of the survey. Of the eight abstracts apparently published in journals as full text, only five resembled the presented abstracts and are included in the preceding analyses. Of the 38 non-published abstracts, 28 (74%) were never submitted for publication, reasons for which are shown in Table 3.
Table 3.
Why abstracts presented at the ASGBI meeting were not submitted for full-text publication
| Reason for non-submission | Number (% of respondents) |
|---|---|
| Lack of time | 8 (27) |
| Pursuit of publication was a low priority | 6 (20) |
| Study was preliminary work for a larger on-going study | 6 (20) |
| Other studies with similar findings were published | 5 (17) |
| Co-investigators left the organisation | 7 (23) |
| Responsibility for writing lay with someone else | 4 (13) |
| The results were not considered sufficiently novel | 4 (13) |
| Statistical analyses were inconclusive | 1 (3) |
| A low likelihood that journals would accept it for publication because of methodological limitations | 8 (27) |
| A low likelihood that journals would accept it for publication because of insufficient interest among readership | 2 (7) |
Discussion
National surgical meetings/conferences serve to disseminate knowledge, crucial to the improvement and maintenance of the standard of care. References in major textbooks contain significant numbers of abstracts.3 However, abstracts lack peer review, contain insufficient information to assess validity and carry a risk of inconsistency with the full-text published manuscript. Abstracts presented to international meetings have subsequent publication rates ranging from 11% to 78%.3–13 This suggests that a large number of abstracts are never published as a complete article. Such failure to publish may limit dissemination of results, which has implications for the quality of research and resources utilised. The unavailability of the complete manuscript may also limit the ability of the reader to assess the reliability and validity of the study and its application to clinical practice.
To our knowledge, this is the first study evaluating the fate of presented abstracts at a national or international conference in general surgical practice. In our study, 56.4% (95% CI 50–62.5%) of the abstracts were subsequently published as a complete report, the vast majority (93.4%) within 4 years of presentation. It is unlikely that we have significantly underestimated the publication rate as we have evaluated publication up to 6 years after the meeting. Studies of orthopaedic and ophthalmology abstracts have shown that more than 90% are published within 4 years of the meeting.5,11 Our search of both PubMed and EMBASE databases also reduces the chances of having missed publications in journals not indexed in either of them. We have, however, assumed that research projects are usually presented at meetings before submission for publication and not the other way around.
It is thought that better quality abstracts are reserved for an oral presentation. Surprisingly, oral and poster forms of presentations did not differ in abstract characteristics, subsequent publication rates, time to publication and inconsistency rates between the presented and published abstract. This finding indicates the need for better measures to define quality and allocation to oral and poster forms of presentation.
In our study, multicentre studies were more likely to be published than single-centre ones (although not statistically significant) and abstracts involving academic centres were significantly more likely to result in publication in a high impact factor journal. Although study design categorised as ‘observational’, ‘interventional’ and ‘others’ did not influence subsequent publication, detailed analysis showed that 10 of the 13 randomised controlled trials (76.9%) resulted in publication. Our data also show that the time taken to publish was significantly influenced by studies involving academic centres (P = 0.034).
Inconsistencies between the presented abstract and the subsequent published manuscript are a serious cause for concern. We found a minor inconsistency rate of 86% and a major inconsistency rate of 6% in our study. Our definitions of ‘minor’ and ‘major’ inconsistencies, albeit arbitrary, reflect the importance of these inconsistencies in determining validity and generalisation of the results. We recognise that a number of minor inconsistencies such as increase in author number, sample sizes, minor changes in results can be considered insignificant, but it serves to highlight the problem of inconsistency in general. Of several factors studied, increased time taken to publish was found to be significantly associated with the presence of inconsistencies between the presented and published abstracts (P < 0.01). We have compared presented abstracts with only abstracts (and not the full content) of published papers and believe that using full text of the published manuscript for comparison with the presented abstract would have increased the rate of inconsistencies.
Several reasons exist for non-publication of abstracts and they include non-submission, publication in other forms (theses, books etc.) and work under progress or under review.4,14 Reasons for non-submission include lack of time, disagreement amongst co-authors, lack of interest in publication, perceived limitations of the study and usefulness of the work to the field.14 Respondents to our questionnaire survey felt that lack of time (27%), methodological limitations (27%), co-investigators leaving the organisation (23%), low priority to publish (20%), preliminary work (20%) and similar published studies (17%) were some of the common reasons for non-submission of manuscripts.
Conclusions
Publication rates and the time taken to publish full-text articles of abstracts presented at a UK national surgical meeting were comparable with results from other similar studies. We have demonstrated that the number of centres and the involvement of academic units may influence this process. Inconsistencies between presented and published abstracts are associated with a longer duration to publish. Various reasons exist for non-submission of abstracts, which need to be addressed to avoid wastage of time and resources.
Acknowledgments
Presented in part to the Travelling Surgical Society, 2003; the Annual Meeting of the Association of Surgeons in Training 2004; and the Annual Meeting of the Association of Surgeons of Great Britain and Ireland 2004.
References
- 1.Narine L, Yee DS, Einarson TR, Ilersich AL. Quality of abstracts of original research articles in CMAJ in 1989. CMAJ. 1991;144:449–53. [PMC free article] [PubMed] [Google Scholar]
- 2.Taddio A, Pain T, Fassos FF, Boon H, Ilersich AL, Einarson TR. Quality of nonstructured and structured abstracts of original research articles in the British Medical Journal, the Canadian Medical Association Journal and the Journal of the American Medical Association. CMAJ. 1994;150:1611–5. [PMC free article] [PubMed] [Google Scholar]
- 3.Bhandari M, Devereaux PJ, Guyatt GH, Cook DJ, Swiontkowski MF, Sprague S, et al. An observational study of orthopaedic abstracts and subsequent full-text publications. J Bone Joint Surg Am. 2002;84:615–21. doi: 10.2106/00004623-200204000-00017. [DOI] [PubMed] [Google Scholar]
- 4.Riordan FA. Do presenters to pediatric meetings get their work published? Arch Dis Child. 2000;83:524–6. doi: 10.1136/adc.83.6.524. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Daluiski A, Kuhns CA, Jackson KR, Lieberman JR. Publication rate of abstracts presented at the annual meeting of the Orthopaedic Research Society. J Orthop Res. 1998;16:645–9. doi: 10.1002/jor.1100160603. [DOI] [PubMed] [Google Scholar]
- 6.Gorman RL, Oderda GM. Publication of presented abstracts at annual scientific meetings: a measure of quality? Vet Hum Toxicol. 1990;32:470–2. [PubMed] [Google Scholar]
- 7.Nguyen V, Tornetta P, 3rd, Bkaric M. Publication rates for the scientific sessions of the OTA. Orthopaedic Trauma Association. J Orthop Trauma. 1998;12:457–9. doi: 10.1097/00005131-199809000-00004. discussion 6. [DOI] [PubMed] [Google Scholar]
- 8.Byerly WG, Rheney CC, Connelly JF, Verzino KC. Publication rates of abstracts from two pharmacy meetings. Ann Pharmacother. 2000;34:1123–7. doi: 10.1345/aph.19375. [DOI] [PubMed] [Google Scholar]
- 9.Scherer RW, Dickersin K, Langenberg P. Full publication of results initially presented in abstracts. A meta-analysis. JAMA. 1994;272:158–62. [PubMed] [Google Scholar]
- 10.Juzych MS, Shin DH, Coffey JB, Parrow KA, Tsai CS, Briggs KS. Pattern of publication of ophthalmic abstracts in peer-reviewed journals. Ophthalmology. 1991;98:553–6. doi: 10.1016/s0161-6420(91)32256-5. [DOI] [PubMed] [Google Scholar]
- 11.Wang JC, Yoo S, Delamarter RB. The publication rates of presentations at major Spine Specialty Society meetings (NASS, SRS, ISSLS) Spine. 1999;24:425–7. doi: 10.1097/00007632-199903010-00002. [DOI] [PubMed] [Google Scholar]
- 12.Hamlet WP, Fletcher A, Meals RA. Publication patterns of papers presented at the Annual Meeting of The American Academy of Orthopaedic Surgeons. J Bone Joint Surg Am. 1997;79:1138–43. doi: 10.2106/00004623-199708000-00004. [DOI] [PubMed] [Google Scholar]
- 13.Jackson KR, Daluiski A, Kay RM. Publication of abstracts submitted to the annual meeting of the Pediatric Orthopaedic Society of North America. J Pediatr Orthop. 2000;20:2–6. [PubMed] [Google Scholar]
- 14.Sprague S, Bhandari M, Devereaux PJ, Swiontkowski MF, Tornetta P, 3rd, Cook DJ, et al. Barriers to full-text publication following presentation of abstracts at annual orthopedic meetings. J Bone Joint Surg Am. 2003;85:158–63. doi: 10.2106/00004623-200301000-00024. [DOI] [PubMed] [Google Scholar]