From Meeting Presentation to Peer-Review Publication – a UK Review

Abstract

INTRODUCTION

Annual academic surgical meetings provide a forum for the discussion of research. For the wide-spread dissemination of this information, peer-reviewed publication is required. The aim of this study was to compare the amount of presentations which go on to publication from 4 UK-based surgical meetings.

MATERIALS AND METHODS

We determined whether a presentation had led to a successful publication using PubMed, a median of 28 months following each meeting. We compared the ASGBI publication rate with the meetings of the Vascular Surgical Society (VSSGBI), the Association of Coloproctology of Great Britain and Ireland (ACPGBI) and the British Transplantation Society (BTS). We also compared the median impact factor of journals used.

RESULTS

The ASGBI and BTS had a similar rate of presentations resulting in publication, with 35% and 36% at 2 years, respectively. The VSS had a significantly greater proportion of presentations resulting in publication (54% at 2 years; P = 0.004), whilst the ACPGBI had significant fewer (24% at 2 years; P = 0.006). There was no difference in the median impact factors of the journals used between the meetings (Kruskal Wallis P = 0.883).

CONCLUSIONS

There is a significant variation between meetings in terms of turning presentations into publications. However, the majority of abstracts have still not been fully published within 2 years of presentation at the meeting.

Academic meetings provide forums for education, discussion and presentation of new work. To enable wide-spread dissemination of research presented at such meetings, full peer-review publication is required. Patients participate in ethically approved trials with the implicit agreement that the study will provide important new knowledge.¹ Furthermore, systematic reviews, which do not include evidence which has been presented but not been formally published (i.e. all available evidence), may be biased or overestimate treatment effects.² For these reasons, it has been suggested that not publishing clinical trials represents scientific and ethical misconduct.¹ However, it is recognised that less than half of the abstracts presented at scientific meetings are likely to be published ever.^3,4

Two recent meta-analyses have identified that there are a number of factors which tend to influence the outcome of presentations. Factors which favour publication include: small meetings; meetings held in the US; oral rather than poster presentation; basic science rather than clinical studies; and studies with positive treatment effects.^3,4 The average time to presentation was 17–20 months.⁴

The aim of this study was to compare the proportion of presentations which were eventually published for four annual scientific surgical meetings held in the UK – the Association of Surgeons of Great Britain & Ireland (ASGBI), the Vascular Surgical Society of Great Britain & Ireland (VSSGBI), the British Transplantation Society (BTS), and the Association of Coloprotocology of Great Britain & Ireland (ACPGBI).

Materials and Methods

Identification of presentations and publications

We identified all the presentations given at each of the four meetings in 2001. Abstract titles and authors were identified using journal supplements for ASGBI, VSSGBI and ACPGBI. For the BTS, the conference proceedings were used as abstracts are not published in any journal. The mode of presentation was also noted (oral or poster).

A median of 28 months (IQR 19–30) following each meeting, we assessed whether the presented abstract had lead to full publication. We searched Medline (PubMed) using the first and last surnames of the authors of each presentation. The title and abstract content of the presentation was then compared with the retrieved results and concordance resulted in the classification of the presentation as published.

Having identified the time of publication and the journal utilised; we used the 2002 impact factor (ISI) as an indicator of journal quality.

Statistics

We compared the oral and poster presentation rate within each meeting for the ASGBI, ACPGBI and BTS (χ² test). No such analysis was performed on the VSSGBI data as there were no poster presentations.

The rates of publication of all presentations and oral-only presentations for each meeting were compared using a Cox proportional hazards model. The ASGBI was used as the indicator (relative risk = 1) and the other three meetings compared to this. Log-minus log plots were used to test the proportionality of hazards assumption.

Median impact factor of journals used for full publication was compared using a Kruskall-Wallis test.

Results

Meeting size and presentation type

Figure 1 shows the number and type of presentations given at each meeting. In terms of number of presentations, the VSSGBI was the smallest meeting and had only oral presentations, whilst the ACPGBI was the largest but had the highest proportion of poster presentations (59.3%).

Figure 1.

The number of oral (open) and poster (hatched) presentations at each of the 2001 annual meetings of the ASGBI, BTS, ACPGBI and VSSGBI.

Effect of type of presentation on presentation within 2 years

We found that there was no difference between publication rates of oral presentations compared to poster presentations for the ASGBI or the BTS. Oral presentations given at the ACPGBI were significantly more likely to result in publication (P < 0.001; χ² test). Table 1 shows a summary of these results.

Table 1.

The number of presentations at each meeting, which have been fully published within 2 years

Meeting	Presentations published			Oral versus poster (χ2)
	All	Oral	Poster
ASGBI (n = 272)	94 (34.6%)	72 (35.8%)	22 (30.9%)	P = 0.461
BTS (n = 132)	47 (35.6%)	28 (41.7%)	19 (29.2%)	P = 0.132
ACPGBI (n = 308)	75 (24.3%)	41 (35.6%)	34 (17.6%)	P = 0.001
VSSGBI (n = 56)	30 (53.6%)	30 (53.6%)	n/a	n/a

n/a, not applicable.

Comparison of rates of publication of the four meetings

Figure 2 shows the publication rates of all presentations at each meeting. At 2 years, the proportion of presentations which had resulted in publication for each meeting was 38% (ASGBI), 39% (BTS), 54% (VSSGBI) and 24% (ACPGBI). Presentations given at the VSSGBI were significantly more likely to result in publication than other meetings. Presentations given at the BTS and ASGBI performed similarly, but were significantly more likely to result in publication than presentations given at the ACPGBI. Table 2 shows the ‘relative risk’ of presentation resulting in publication at each meeting with 95% confidence intervals.

Figure 2.

Kaplan-Meier survival curves to show rates of publication for all presentations at each of the four meetings. Number of abstracts remaining ‘at risk’ at each time point is shown in the table.

Table 2.

Relative risks of full publication of all abstracts following presentation at each meeting compared to the ASGBI using Cox proportional hazards regression model

Meeting	Relative risk of publication	95% confidence intervals	P-value
ASGBI	1	n/a	n/a
BTS	1.096	0.786–1.529	0.588
ACPGBI	0.653	0.487–0.875	0.004
VSSGBI	1.782	1.183–2.684	0.006

n/a, not applicable.

Table 3.

Relative risks of full publication of abstracts following oral presentation at each meeting compared to the ASGBI using Cox proportional hazards regression model

Meeting	Relative risk of publication	95% confidence intervals	P-value
ASGBI	1	n/a	n/a
BTS	1.157	0.760–1.761	0.497
ACPGBI	0.964	0.667–1.393	0.844
VSSGBI	1.677	1.098–2.561	0.017

The differences between the meetings of the ACPGBI, ASGBI and BTS disappear when only oral presentations are compared (Fig. 3). However, the conversion rate of oral presentation to publication at the VSSGBI was still significantly betterable 3).

Figure 3.

Kaplan-Meier survival curves to show rates of publication for oral presentations at each of the four meetings. Number of abstracts remaining ‘at risk’ at each time point is shown in the table. Oral presentations given at the VSSGBI are significantly more likely to be published.

Comparison of journals used for publication

Table 4 shows the total number of journals used per meeting, and the most commonly used journal per meeting. The average impact factor of journals used for publication did not vary significantly between meetings (Fig. 4).

Figure 4.

Comparison of average journal impact factor used for eventual publication of presentations from each meeting. There is no significant difference between the meetings (Kruskal-Wallis P = 0.883).

Table 4.

A summary of the journals in which presentations are eventually fully published

Meeting	Total no. of public.	Total no. of journals	Most commonly used journal
ASGBI	101	45	British Journal of Surgery (n = 17)
BTS	54	20	Transplantation (n = 19)
ACPGBI	81	26	Diseases of Colon & Rectum (n = 25)
VSSGBI	30	8	European Journal of Vascular & Endovascular Surgery (n = 14)

Discussion

We have found significant variation in the conversion of presentations to publications between four academic surgical meetings in the UK. One explanation is that the work presented at certain meetings is, overall, of lower quality than work presented at others. Alternatively, it could be argued, on this evidence, that vascular surgeons are better at assessing abstracts and writing manuscripts than their general surgical colleagues. We believe these explanations are too simplistic and probably unlikely. The ASGBI and BTS perform similarly, with 35% and 36% of abstracts published fully within 2 years and this compares well with publication rates determined by a meta-analysis using a similar survival analysis (27% at 2 years).³ Presentations at the VSSGBI are far more likely to result in publication. This is probably due to the fact that there are a relatively small number of abstracts which are all presented orally, both factors which have been identified as positive predictors for publication.³ Having only a small number of presentations may make the abstract submission more competitive and result in presentation of work more likely to be published. The ACPGBI performs relatively badly due to the large number of poster presentations: oral presentations at the ACPGBI were just as likely to result in presentation as the BTS or ASGBI. Interestingly, the meetings compared well in terms of the impact factor of the journals in which presentations were eventually published.

Our study has looked at only one year for each meeting and, as such, gives a snap shot of how the meetings were functioning as a forum for presentation of research that would eventually reach full publication at that time. Furthermore, although we have examined the rates of publication after the point when most of what will eventually be published has made it into print,^3–5 it is possible that with longer follow-up the meetings may become more homogeneous. Although we have examined the influence of the mode of presentation (oral versus poster), we have not attempted to examine further the differences in types of presentations between each meeting, such as basic versus clinical science or randomised studies versus other trial designs. It is conceivable that there may differences between each meeting in terms of the mix of these types of study which will make publication more likely.

The main finding of concern is that the rate of eventual publication is still low – a large proportion of what is presented has not been published. There may be a number of reasons for this. First, it is more difficult to peer review abstracts effectively than full manuscripts; therefore, studies with poor designs and questionable reliability^6,7 may make it to presentation but not publication. However, effective meetings not only serve as platforms to present data but also to provide a form of informal peer review. Studies which are poorly designed, performed or reported may be filtered out by criticisms at such meetings. However, any clinical study which has been conducted with ethical approval should have been given a formal review with regards to the need for the study and its methodology prior to its commencement.⁸

Whilst stringent peer-review publication should be up-held, it is important that any barriers to publication which do not relate the quality of the study are reduced. The majority of presentations are undertaken by trainees. Due to the nature of clinical practice, the writing of manuscripts will usually be performed out of work and may become a low priority when compared other to commitments. Indeed, lack of time was the most frequently quoted reason for non-publication in a survey of orthopaedic trainees.⁹ Whilst manuscripts require time, re-drafting and often lengthy submission processes, a presentation can be attained relatively quickly. Presentations may serve as a defined, achievable goal for a trainee within a year-long post, for instance. If the motivation to present work is merely to fulfil a goal at one particular time, it is unsurprising that work does not get published. Furthermore, large prospective or randomised studies with appropriate follow-up and sufficient power are virtually impossible to perform over a short period of time. Setting short-term goals for trainees is probably not effective at producing studies of this type, and thus results in easily performed retrospective studies which may prove more difficult to publish than present.

Given the general concerns over the reliability of results in abstracts^6,7 and that not including evidence that has not been fully-published in systematic reviews may introduce bias,^1,2 perhaps it should be made more difficult present work at such meetings. Possibly the most effective strategy to improve rates of publication would be to limit the amount of work accepted – as demonstrated by the VSSGBI in our study. The changes in surgical training¹⁰ may also have an impact on the amount, as well as the quality, of research performed by trainees. Whilst many trainees currently undertake a period of research either within or prior to commencing higher surgical training,^11,12 there is no obvious point within the proposed structure of seamless surgical training at which this can be done. It could be argued, however, that only the highly motivated future academics will undertake research and would, therefore, be more likely to see the process through to publication. Alternatively, it could mean that meetings will be filled with small retrospective studies which are unlikely to be published. We await the effect the changes will have with interest.

As meta-analysis and systematic reviews aim to evaluate all available evidence, it is imperative at the very least that it is possible to identify all research that has been undertaken into a subject, including that which has not been published fully. The so-called ‘grey literature’ (unpublished) is clearly difficult to identify.² Whilst conference proceedings and journal supplements are available, hand searching for subject areas is usually required. ISI Web of Knowledge now indexes abstract titles individually for many journals, including the British Journal of Surgery, making searching easier and more effective. We believe that conference organisers have a responsibility to ensure that the meetings' abstracts are available and, ideally, published in fully indexed journals. In this way, duplication of studies may be prevented and the grey literature identified efficiently.

Conclusions

There is a significant variation between the meetings we have studied, in terms of turning presentations into publications. Worryingly, the majority of presentations have not been published within 2 years of the meetings. Ensuring that abstracts are of high quality and available to those not attending the meeting is essential so that all work contributes to the ‘available evidence’ and studies not duplicated unnecessarily.