Publication Outcome for Research Presented at the Vascular Society of Great Britain and Ireland Annual Meetings

Abstract

BACKGROUND

The Vascular Society of Great Britain and Ireland (VSGBI) annual meeting is a major international vascular surgery conference. Studies suggest that the percentage of presentations that result in full-text publications are a measure of the quality of the meeting. We investigated the publication outcome of abstracts presented to the VSGBI in 2001 and 2002.

MATERIALS AND METHODS

We retrospectively identified abstracts from the conference programmes and conducted a detailed electronic Medline and PubMed search to determine publication. We collected data regarding the study design, subject matter, publishing journal, time to publication, institution of origin, impact factors and RAE levels.

RESULTS

There were 63 publications from 106 abstracts (59.4%), with a median impact factor of 3.507. Prospective observational studies accounted for 20.6% of publications, with abdominal aortic aneurysms being the commonest subject matter (34.9%). The median time to publication was 12 months, with the European Journal of Vascular and Endovascular Surgery publishing 33.3% of the articles. Leicester achieved the highest number of publications and the majority of work came from centres with Research Assessment Exercise (RAE) level scores of 4, university centres accounted for 74.6% of publications.

CONCLUSIONS

We conclude that when compared to equivalent meetings in other specialties and geographical regions, the annual meeting of the VSGBI is of the very highest quality.

Presentations given at annual meetings of surgical societies allow the re-evaluation and modification of current surgical techniques and clinical practice, and dissemination of new scientific research. These presentations also hold significant importance in the career progression of surgical trainees.

It has been suggested that the ultimate goal of any research is to provide a long-lasting, retrievable record of the work in the form of a full-text published article;¹ furthermore, studies have suggested that the quality of a meeting can be assessed by the number of presented abstracts that result in a full-text publication in a peer-reviewed journal.^2,3 The Vascular Society of Great Britain and Ireland (VSGBI) Annual General Meeting maintains its position as the premier vascular surgical meeting nationally and consistently attracts eminent international speakers. Our aim was to assess the percentage of abstracts presented at the 2001 and 2002 VSGBI annual meetings that achieved full-text publication. In addition, we aimed to determine the type of study, subject matter, and institutions that were successful in achieving publication.

Materials and Methods

All the abstracts presented at the VSGBI annual meetings in 2001 and 2002 were identified retrospectively from the VSGBI yearbook that also acts as a conference programme to accompany the meeting.

To determine whether or not the abstract had been published, a detailed electronic literature search was made of the Medline database limited from January 1999 to April 2005. The search was initially conducted for the first author; if this was unsuccessful, searches were then conducted using the remaining authors and appropriate key words from the title. If the Medline search was unsuccessful, a second search using the above criteria was performed on the PubMed database (<www.ncbi.nlm.nih.gov/entrez/query.fcgi>).

All the presented abstracts from the VSGBI annual meeting are published in the British Journal of Surgery. This study was conducted to quantify full-text publications arising from the meeting; therefore, these abstract publications have not been included in the data analysis.

Once a publication had been identified, information was recorded as to the study design and subject matter, the institution in which the work had been undertaken, the journal in which the work was published, and the time taken from presentation to publication. Following this, the journal impact factor of the publishing journal was derived from Journal Citation Report produced by the Thomson Institute for Scientific Information. As the meetings being studied were 2002 and 2001, the impact factors for 2002 were used and accessed through <www.bioreference.net/impact>. Finally, the 2001 Research Assessment Exercise (RAE) level scores for each institution were noted where available (<www.rae.ac.uk>). The allocation of RAE levels are a peer-review exercise to evaluate the quality of research in higher education institutions within the UK. The levels, which range from 1 (equating to virtually no research to a level of national excellence) to 5* (equating to international excellence in over half of the research activity, with the remaining half being at a level of national excellence), endow a measure of quality on an institution (<www.rae.ac.uk>).

We did not contact the authors of unpublished abstracts to enquire as to reasons for non-publication as this has been performed by a number of other investigators. In addition, we did not perform a hand search of any non-cited journals as this would have proved overly time consuming for a potentially very modest increase in data accuracy.

Descriptive statistics were used in the analysis of the collected data.

Results

The selection process employed in accepting abstracts submitted to the VSGBI for presentation will not be addressed in this study. However, as a guide to the volume of abstracts submitted for review, and the selection process, the following information was collected from the VSGBI. For the 2001 and 2002 meetings, 187 and 264 abstracts were submitted, respectively. The abstracts were independently scored by 9 council members on criteria of originality, clinical importance and scientific method. A final score was given and cut off for acceptance determined. This resulted in 57 and 49 abstracts being accepted for the 2001 and 2002 meetings, respectively. In total, 23.5% of the submitted abstracts being accepted for publication. To illustrate further the selection process, in 2002 the 264 abstracts submitted achieved scores from 2.7–8.0 out of 10, and the 49 accepted abstracts scored from 6.0–8.0.

Publication outcome

In total, at the VSGBI annual meetings of 2001 and 2002, there were 106 abstracts selected for oral presentation. This resulted in 63 full-text publications in peer-review journals, giving a publication rate of 59.4%. There was no evidence of dual publication and no papers were seen to combine the results of two or more abstracts.

Study subject-matter and design

The published studies covered a large range of specific areas of interest to vascular surgeons as outlined in Table 1. Specifically, the commonest topics to be accepted to a journal were 8 (12.7%) publications addressing the operative treatment of elective and ruptured abdominal aortic aneurysms (AAAs), 7 (11.1%) publications regarding the endovascular treatment of AAA and 7 (11.1%) publications studying carotid endarterectomy. There we many other topics successful in gaining publication but examples of those less covered are, surgical training, out-of-hours interventional radiology availability and cold provocation testing.

Table 1.

The specific subject matter of published abstracts

	Publications
Subject matter	No. of	% of
AAA screening	3	4.8
AAA expansion/rupture factors	4	6.4
AAA endovascular treatment	7	11.1
AAA elective and rupture open treatment	8	12.7
Infra-inguinal bypass surgery	5	7.9
PVD risk factors	1	1.6
PVD diagnosis	1	1.6
PVD non-surgical management	2	3.2
CLI non-surgical management	1	1.6
Carotid endarterectomy	7	11.1
Carotid disease	3	4.8
Carotid angioplasty	1	1.6
Varicose vein surgery	2	3.2
Venous ulcer surgery	1	1.6
Venous thrombosis	1	1.6
Angiogenesis	3	4.8
Thrombolysis	2	3.2
Mortality data/league tables	1	1.6
Popliteal aneurysm treatment	1	1.6
Cold provocation test	1	1.6
Stump dressings	1	1.6
Thoracic aneurysm endovascular treatment	2	33.2
Surgical training	1	1.6
Infection	1	1.6
False femoral aneurysm treatment	1	1.6
Renal artery stenosis	1	1.6
Interventional radiology out-of-hours availability	1	1.6

When classifying the topics into broader subsets, the most commonly published topic was AAA disease with 22 (34.9%) publications, followed by 13 (20.6%) articles concerning peripheral vascular disease (PVD), and 11 (17.5%) papers concerning carotid disease. The general subsets of the published study subject matter can be seen in Figure 1.

Figure 1.

Graphical representation of the general subject matter of published abstracts.

A variety of study designs were used in the abstracts that were successfully published and these are outlined in Table 2. The 63 publications were mainly made up of 13 (20.6%) prospective observational studies, 9 (14.3%) basic science research papers, 8 (12.7%) randomised control trials, and 8 (12.7%) papers from prospective registry data such as the Eurostar database and UK small aneurysm trial.

Table 2.

Study designs that were published

	Publications
Type of study	No. of	% of
Randomised control trial	8	12.7
Prospective observational study	13	20.6
Prospective registry data	8	12.7
Retrospective hospital statistics' analysis	1	1.6
Basic science research	9	14.3
Comparative imaging study	1	1.6
Audit	5	7.9
Case-control	2	3.2
Comparative PVD assessment study	1	1.6
Retrospective observational study	2	3.2
Retrospective case note review	4	6.4
Retrospective data analysis	2	3.2
Epidemiology	2	3.2
Meta-analysis	1	1.6
Postal survey	1	1.6
Comparative PVD treatment study	1	1.6
Literature review	1	1.6
Retrospective cost analysis	1	1.6

Time to publication, journals and impact factors

The median time from presentation to publication was 12 months (interquartile range, 7–18 months), with 6 (9.5%) papers being published prior to the conference. The longest time from presentation to publication was 32 months. The mean was 11 months.

The full-text publications that followed presentation appeared in 15 different peer-review journals. The journals that accepted the majority of the studies for full-text publication were the European Journal of Vascular and Endovascular Surgery, which published 21 (33.3%) of the studies, the British Journal of Surgery and the Journal of Vascular Surgery which published 18 (28.6%) and 9 (14.3%) papers, respectively. The remaining journal information is represented in Figure 2.

Figure 2.

The published papers by journal

The median journal impact factor for the full-text publications resulting from the presentations was 3.507 (interquartile range, 1.774–3.772), with 18.32 being the highest attained for the Eschar venous ulcer study published in The Lancet.

Institutions and RAE levels

The final 63 publications came from 32 different centres, with the most arising from Leicester, 8 (12.7%) publications, Liverpool, 7 (11.1%) publications, and Guys, Kings and St Thomas's, 6 (9.5%) publications. The centres with university affiliations accounted for 47 (74.6%) of the total publications arising from the presentations.

RAE levels are only allocated to higher institutions receiving funding from the Higher Education Councils of England, Scotland, Wales and Northern Ireland; therefore, it was impossible to allocate scores to papers arising from centres with no university affiliation or those not listed. We were able to allocate levels to centres accounting for 40 (63.5%) of the 63 publications. For these 40 papers, 5 (12.5%) came from institutions with a 5* rating, 8 (20%) came from a level 5 institution, 26 (65%) came from a level 4 institution, and finally 1 (2.5%) came from a level 3a institution.

Discussion

The meetings of surgical societies and groups are essential in the dissemination of current knowledge and research. They are also particularly important in the career progression of surgical trainees. Studies have suggested that the number of full-text publications in peer-review journals is a good indicator of the quality of the meeting.^2,3 On review of the abstracts presented at the VSGBI 2001 and 2002 annual meetings, 59.4% resulted in publications. Although this figure is probably an underestimation, due to the limitations of the study which will be discussed later, when compared to meetings from other specialties, both within Britain and internationally (Table 3), this is a relatively high publication rate.

Table 3.

Comparison of publication rates from scientific meetings

Author	Specialty	Region	Publication rate
Bhasin & Scott (this study)	Vascular surgery	Great Britain and Ireland	59.4%
Scherer et al.5	Meta-analysis of 11 papers	Meta-analysis	51%
Scherer et al.5	Ophthalmology	USA	66%
Riordan9	Paediatrics	Britain	51% and 78%
Walby et al.4	Emergency medicine	Australasia	35%
Callaham et al.10	Emergency medicine	USA	43%
De Bellefeuille et al.11	Oncology	USA	58%
Yoo et al.13	Orthopaedics	USA	50.9% and 68.1%
Wang et al.3	Spine	USA	43.5%
Yentis et al.7	Anaesthesia	Canada	50%
Bowrey et al.1	Surgery	Wales	47%
Bhandari et al.8	Orthopaedics	USA	34.2%

Although the use of publication rate may be a crude measure of quality. it does allow comparison across specialties and regions.⁴ Wang et al.³ conducted a review of abstracts presented at the North American Spine Society, Scoliosis Research Society, and International Society for the Study of the Lumbar Spine meetings and calculated a publication rate of 43.5%. They concluded this reflected the high quality of the meetings and validated their selection procedures.³ In addition, a meta-analysis of 11 reports covering 7 specialties calculated a mean publication rate of 51% from these national meetings.⁵ In light of this evidence, the VSGBI publication rate of 59.4% demonstrates the value of this meeting and the selected presentations in an international, multidisciplinary setting.

We also believe that this figure represents the minimum output from the meeting due to the limitations of the study. First, as we did not conduct a hand search or contact the authors regarding publications, we were limited by the electronic literature search. Any publications in journals not cited on the index will have been omitted from the data. It has also been noted that to gain the most accurate results from the search engines, the investigator must have a detailed knowledge of the structure of the database and how it is indexed.⁶ A second limitation may have been the timing of the search; a number of studies have reported that at least 88% of the full-text manuscripts arising from a meeting are published within 3 years.^4,5,7 The search we performed was 29 months after the 2002 meeting. In reviewing the figures for the 2001 meeting, we noted 3 publications arising after 29 months and this indicates that there may still be articles in-press or to be accepted for publication after our search. The final limitation we encountered was the definitive identification of an article. Bhandari et al.⁸ followed up 465 abstracts presented to the 1996 Annual Meeting of the American Academy of Orthopedic Surgeons and noted a number of inconsistencies between the accepted abstracts and subsequent publications. They revealed a change in title in 43.4%, change of authorship in 29.6%, change of primary outcome in 13.2%, change of sample size in 8.8%, change in results in 18.9%, and change in statistical methods in 8.2% of publications when compared to original abstract.⁸ This observation highlights the difficulty of precisely identifying any resultant publications. Where there was any discrepancy, it was left to the discretion of this study's first author as to the inclusion or exclusion of any non-identical, but similar, articles that were discovered on the electronic search. The same principle applied to work from Data Registries presented to the VSGBI that may have been included as part of a larger publication at a later date. Unless there was a specific publication relating to the presentation, it was not included in the final analysis; this may have led to a further slight underestimation of the VSGBI publication rate. A study which used a similar search method and surveyed authors believed they underestimated the true publication rate by 15.5%.⁶ Should that be the case in our study, the actual publication rate from the VSGBI may be as high as 74.9%.

The specific subject that resulted in most publications was open surgical treatment of elective or ruptured AAAs (12.7%) with the endovascular treatment of AAA (11.1%) being the next common. This obviously led to AAA also being the most published subject in broad terms as well accounting for 34.9% of full-text articles. This re-inforces the importance the vascular surgical community places on the need for advancement and improvement of treatment in this grave condition. As we believe there have been no other investigations into publications resulting from vascular meetings, there are no comparable figures.

With regard to study design, the publications were mainly made up of prospective observational studies (20.6%), basic science research papers (14.3%), randomised control trials (12.7%), and prospective registry data (12.7%). Some studies have observed that randomised control trials are published more often, citing the time and effort taken in conducting the trial as the main reason for acceptance.^5,9 Our study corroborates the data from Bhandari et al.⁸ concluding observational studies are dominant in progressing from presentation to publication.

We found that the majority of presentations that resulted in publication appeared in the European Journal of Vascular and Endovascular Surgery (33.3%), and the mean time to publication for all abstract was 11 months. One may expect this journal to be the most popular choice as it constitutes the most prominent ‘local’ specialist vascular publication. This mean time to publication is faster than full-text publications given at a number of other meetings where the range was 12.6–20 months.^1,4,8,10,11 Where a presentation resulted in a full publication, the median impact factor attained was 3.507, as compared to 1.48 for publications arising from presentations at the 1991 annual US research meeting for the Society of Academic Emergency Medicine.¹⁰ The finding of 74.6% of publications arising from centres with university affiliations may not be an unsurprising fact. Sprague et al.⁶ found abstracts presented to a national American Orthopedic Conference from an academic centre were significantly more successful in attaining full-text publication as compared to those from non-academic centres (76% versus 17%; P < 0.01).

These data highlight the issues regarding the journal of choice for conference abstract and full-text publications arising from the VSGBI. Historically, the British surgical societies have been allied to the British Journal of Surgery and this accounts for the appearance of the conference abstracts within this journal. However, with the aim of increasing the impact factor of the British Journal of Surgery, this may in time lead to the removal of specialist society abstracts from the journal and their being replaced by electronic publication as supplements. In the case of the European Journal of Vascular and Endovascular Surgery, it may be inappropriate to publish the abstracts from each individual European member state's national vascular meeting in a European journal.

Where the publication of high-quality, basic science research and large-scale RCTs are concerned, the impact factor of the journal will always remain a deciding factor. The review of grant applications to support future basic science projects, and the peer-review RAE rating, necessitates that any full-text publications resulting from a unit's basic science project must be published in the journal with the highest possible impact factor, whether or not the article is completely suitable for that journal's readership. The VSGBI recognised submitted abstracts with a high basic science content may have been disadvantaged, possibly by the selection process. The potential for failing to attract basic science papers to the annual meeting led to a link with the Society of Academic and Research Surgery (SARS), thus enabling a joint parallel academic vascular medicine session at the annual meeting. Currently, the VSGBI only requires submission of an abstract for presentation in comparison to the meeting of the ESVS, where a presentation must be accompanied by a manuscript ready for publication. In view of the point regarding the journal impact factor discussed above, this may restrict the type of study submitted for presentation at the European Society of Vascular Surgery (ESVS), where the VSGBI in conjunction with SARS would continue to ensure presentations are diverse and of a high quality, as resulting full-text publications are not restricted to one certain journal.

We did not feel it necessary to survey authors who were unsuccessful in publishing as this is well documented in the literature. The main reason cited for non-publication after a presentation is lack of time to prepare a manuscript, which accounts for 29.5–56% of cases.^5,6,11,12 Surveys of authors also discovered other common reasons for non-publication were: (i) the work was still in progress (11–31%); (ii) there had been problems with the co-authors (9–36.6%); (iii) publication was never a priority (12.7–20.5%); (iv) it was unlikely it would be accepted (12.7–20%); (v) there was similar work already submitted (4.2–6%); and (vi) it had been submitted but was rejected (9.1–16.1%).^5,6,11,12 The factors that were seen as essential for successfully publishing in a full-text journal after a presentation at a national meeting was: (i) adequate time for the primary investigator to write up the findings; (ii) a core group interested in co-investigating the subject; (iii) a review of the literature to establish the context of the study and sound methods; and (iv) co-operation with allied academics such as statisticians.⁶

Conclusions

We have defined the type of study subject and institution that is successful in attaining publication after presenting to the VSGBI. We have also demonstrated that the majority of presentations given to the society attain full-text publication in high impact peer-review journals in less time than those given at other conferences. We believe our study demonstrates the annual meeting of the VSGBI to be of a very high quality when compared to equivalent meetings for different specialties, in different geographical regions. This investigation could provide a benchmark for future work to allow comparison in outcome of other major international meetings held within the UK and Europe in vascular surgery, general surgery and other specialties.