Publication rate of abstracts presented at the Annual Congress of the Spine Society of Europe (years 2000–2003)

Abstract

Introduction

The quality of presentations at medical conferences is of major importance. The publication rate (PR) following congress presentation is an indicator of the extent and quality of a scientific society’s activity. The purpose of this study was to investigate publication rates in the Spine Society of Europe (SSE), compare them with the results for American spine societies, and determine factors affecting publication.

Materials and methods

All 839 abstracts of podium and poster presentations at SSE congresses held in 2000–2003 were investigated. PRs in peer-reviewed journals within a period of 5 years were assessed. Subgroup analyses were performed for different study types. The consistency of abstracts with publications was also analyzed.

Results

The overall PR was 37.8%, with a mean of 17.7 ± 15.7 months between congress and publication and a mean impact factor of 1.8 ± 1.0 at the time of publication. Comparatively high PRs were found for podium presentations versus posters, studies with higher versus lower levels of evidence, experimental versus clinical studies, prospective versus retrospective studies, randomized versus nonrandomized studies, studies reporting significant main results versus those without, and multicenter studies versus single-center studies. Biomechanical studies also achieved high PRs.

Conclusion

The PR was similar to that of NASS (40%) and only slightly inferior to that of SRS (47%) and ISSLS (45%). This shows the high quality of presentations at SSE congresses. The fate of unpublished abstracts is worth further consideration. It is questionable whether it is acceptable to cite abstracts that have not passed a journal’s peer-review process and to implement their results in clinical practice.

Introduction

The ultimate goal for research groups is to present their results to the public. Two principal ways of doing this are presentation at conferences and publication in peer-reviewed scientific journals. Publication in a peer-reviewed journal is considered to be the gold standard for disseminating scientific data. In contrast to that type of publication, which includes a full peer review of a complete manuscript with every detail of the study being assessed, conference abstracts that are submitted to a society prior to the meeting are short and can therefore only include basic data. Podium or poster presentations at congresses are also limited in terms of time—often only a few minutes—and in the extent of the data displayed. Despite this, congress presentations are often referred to and cited in everyday clinical decision-making in routine patient care, clinical guidelines, textbooks, medical education, medical science, and even scientific publications [5, 12]. They therefore have a huge impact. The quality of the abstracts presented is thus of major concern in the medical field with which the congress is concerned. The rate of publication of the studies presented in conference abstracts following the meeting is one of the broadly accepted quality features of a congress, indicating the scientific value of the studies presented.

Several medical societies have been investigating the publication rates associated with their major conferences; those in the field of the spine and orthopedics are presented in Table 1. These studies have been routinely published in high-quality peer-reviewed journals. The results provide important quality-control data for the societies involved in the congresses investigated. They may help to maintain and even improve the scientific quality of congress presentations.

Table 1.

Literature review: publication of congress abstracts from specific spine conferences, general orthopedic conferences, and specific orthopedic conferences

Society	Publication rate during follow-up (%)	Year	Follow-up	Conference abstracts included (n)	Podium/poster presentation	Mean time to publication
I. Specific spine conferences
North American Spine Society [27]	40.0	1990–1992	5–7 y	545	Pod + post	n.c.
Scoliosis Research Society [27]	47.4	1991–1993	4–6 y	308	Pod + post	n.c.
International Society for the Study of the Lumbar Spine [27]	45.4	1991–1993	4–6 y	335	Pod + post	n.c.
II. General orthopedic conferences
American Academy of Orthopaedic Surgeons [14]	46	1990–1992	4–6 y	1465	Podium	20 mo
American Academy of Orthopaedic Surgeons [22]	44	1993	5 y	573	Podium	n.c.
American Academy of Orthopaedic Surgeons [5]	34	1996	5 y	465	n.m.	17.6 mo
American Academy of Orthopaedic Surgeons [15]	55	1999	5 y	318	n.m.	n.c.
American Academy of Orthopaedic Surgeons [9]	49	2001	5 y	756	Pod + post	n.c.
	52	2001	5 y	288	Podium	n.c.
	47	2001	5 y	468	Poster	n.c.
Australian Orthopaedic Association [16]	31	1998	5 y	200	Podium	n.c.
British Orthopaedic Association [6]	57	1980–1984	Min. 5 y	320	n.m.	21.4 mo
	45	1990–1994	Min. 5 y	685	n.m.	16.8 mo
British Orthopaedic Association, British Association for Surgery of the Knee, British Orthopaedic Foot Surgical Society, British Elbow and Shoulder Society [13]	33	1997–1998	Not clear	415	n.m.	15.6 mo
Orthopaedic Research Society [7]	52	1991–1993	4–6 y	888	Podium	18–23 mo
III. Specific orthopaedic conferences
American Association of Hip and Knee Surgeons [21]	58	1996–2001	5–10 y	292	Podium	21.7 mo
American Orthopaedic Society for Sports Medicine [30]	66.9	1990–1993	5–8 y	166	Pod + post	n.c.
American Orthopaedic Society for Sports Medicine [19]	59.4	1999–2001	5 y	165	Podium	21 mo
American Society for Surgery of the Hand [11]	52	1991–1992	3–5 y	397	Podium	n.c.
Arthroscopy Association of North America [30]	46.1	1991–1993	5–7 y	167	Pod + post	n.c.
Hip surgery-related abstracts:
British Hip Society [28]	19.7	2004–2005	Mean 38 mo	163	Podium	4.9 mo
European Hip Society [28]	28.5	2004, 2006	Mean 38 mo	241	Podium	4.9 mo
British Orthopaedic Association [28]	23.5	2004–2006	Mean 38 mo	95	Podium	4.9 mo
European Federation of Orthopaedics and Traumatology [28]	22.0	2003, 2005	Mean 38 mo	639	Podium	4.9 mo
International Society of Arthroscopy, Knee Surgery and Sports Medicine [10]	37.0	1997, 1999	4–6 y	358	n.m.	n.c.
Musculoskeletal Tumor Society [18]	41	1991, 1992, 1995, 1997–1999	3–11 y	336	Podium	21.8 mo
Orthopaedic Trauma Association [23]	64	1990–1994	3–7 y	429	Podium	16 mo
Orthopaedic Trauma Association [25]	67	1994–1998	6–10 y	329	Podium	24.8 mo
				486	Poster	21.6 mo
	52	1994–1998	6–10 y
Orthopaedic Trauma Association [24]	67	1994–1997	6–10 y	254	Podium	24.8 mo
Pediatric Orthopaedic Society of North America [17]	53	1991–1994	4–7 y	349	Podium	29 mo
Shoulder and Elbow Session of the American Academy of Orthopaedic Surgery [8]	58	1999–2004	3–8 y	558	Pod + post	31 mo
	66	1999–2004	3–8 y	233	Podium	31 mo
	51	1999–2004	3–8 y	325	Poster	31 mo
Trauma Sessions of the European Federation of National Associations of Orthopaedics and Traumatology [20]	40.3	1999, 2001	5–7 y	278	Podium	25.3 mo

Min. minimum, mo months, n.c. not calculated, n.m. not mentioned, Pod + post podium and poster, y years

Only one investigation on this topic has been published in the growing field of spine surgery [27]. There have been no studies evaluating what happens to abstracts after presentation at the Spine Society of Europe (SSE) conference. The SSE has become one of the world’s most important spine societies, with a growing membership and increasing activities. Each year, the annual SSE congress is one of the highlights for spine specialists, substantially influencing spine surgery throughout the world. The 2010 congress in Vienna, for example, had a total attendance of 3,490 from 82 countries represented—2,198 delegates and 1,292 booth personnel. The importance of the SSE therefore justifies a comparable systematic study of its congress abstracts.

The purpose of this study was to investigate the publication rate of studies presented at the annual SSE congress and to evaluate factors that might help predict peer-reviewed publication. In addition, the consistency of congress abstracts with subsequent publications was assessed. The main hypothesis was that the publication rate would be similar to that of comparable spine congresses [North American Spine Society (NASS), 40.0%; Scoliosis Research Society (SRS) 47.4%; and International Society for the Study of the Lumbar Spine (ISSLS), 45.5%] [27].

Materials and methods

A review of the literature shows that a follow-up period of 5 years is an established value for investigating publication rates, as more than 90% of the published abstracts achieve publication within 4 years following congresses [7, 9, 14, 16, 19, 27, 30]. This study was carried out in the summer of 2009. To allow a follow-up of 5 years, the annual SSE congresses held in 2000–2003 were included.

All 839 abstracts of podium and poster presentations in the years 2000 (n = 259), 2001 (n = 213), 2002 (n = 168), and 2003 (n = 199) as published in the European Spine Journal were studied [1–4]. A PubMed search (MEDLINE) for matching peer-reviewed publications was conducted, covering a 5-year follow-up period after each congress. In addition, publications of abstracts published prior to the congress were also included.

The names of each abstract author were searched first. If multiple publications were found for one author, keywords for the abstract were added to the search. At least one author of the abstract and publication had to be identical for a positive match to be registered.

The congress abstract’s content was directly compared with the publication’s content. If the study hypothesis, methods, sample size, and results were identical, the abstract was graded “published.” If the publication had smaller or larger sample sizes, the corresponding abstract was defined as “published” only if it had an identical hypothesis and methods. In the case of multiple publications per abstract, that publication was chosen for which the publication date was closest to the congress. If a publication was found before the congress, the abstract was graded as “published” if the sample size and follow-up period were identical.

The month of print publication was defined as the date of publication, as the date of acceptance of a manuscript and online publication dates were not available for all of the studies. Differences between the congress and publication dates were assessed in full months. For each publication, the name and impact factor of the journal in the year of publication were noted.

All of the abstracts were classified according to several features in order to determine which abstracts were more likely to achieve full publication in comparison with others. The type of study was assessed (experimental studies, clinical studies, epidemiologic studies, reviews, and case reports). The clinical studies were further subclassified as therapeutic, prognostic, diagnostic, or economic. The level of evidence (LoE) was calculated for clinical studies [29].

It was investigated whether the following study characteristics of congress abstracts had an effect on publication rates in peer-reviewed journals: randomized versus nonrandomized studies (only applying to clinical and epidemiological studies); prospective versus retrospective studies (only applying to clinical and epidemiological studies); single-center studies versus multicenter studies (only applying to clinical studies); biomechanical studies versus nonbiomechanical studies; studies reporting a significant main result versus those without (applying to all studies). If an abstract reported a result at P < 0.05, or if significance was explicitly stated, it was graded “significant.” A result was graded as “not significant” if P > 0.05, or if it was explicitly described as “not significant,” or if significance was not mentioned.

The consistency of the congress abstracts with the corresponding publications was investigated, including the composition of the author groups, sample sizes, and the main results and conclusions. Author groups were studied with special emphasis on the main authors (first or last author). “Identical composition of authors” also included cases in which the abstract authors did not appear as the publication authors, although without any additional authors in the publication. Different orders of main authors and coauthors were accepted in this group. Sample sizes were graded as “same” (with at most ±5% deviation), “smaller,” or “larger.”

Statistical analysis was performed using SPSS 18.0 (SPSS Inc., Chicago, Illinois, USA; 2009). Descriptive analysis was performed. The Chi-squared test was used to compare categorical variables in the study groups. Odds ratios (OR) were calculated with 95% confidence intervals (CI). Differences were considered significant at P < 0.05.

Results

A total of 839 abstracts were studied, including 318 podium presentations (37.9%) and 521 poster presentations (62.1%), with an overall publication rate of 37.8% (Table 2).

Table 2.

Publication rates, 2000–2003

Year	Congress abstracts (n)	Podium presentations (n)	Poster presentations (n)	Published abstracts (n)	Publication rate (%)	Published podium presentations (n)	Publication rate of podium presentations (%)	Published poster presentations (n)	Publication rate of poster presentations (%)
2000	259	94	165	78	30.1	42	44.7	36	21.8
2001	213	80	133	86	40.4	37	46.3	49	36.8
2002	168	72	96	68	40.5	38	52.8	30	31.3
2003	199	72	127	85	42.7	37	51.4	48	37.8
Overall	839	318	521	317	37.8	154	48.4	163	31.3

There was an increase in the publication rate between 2000 and 2003 (from 30.1% in 2000 to 40.4% in 2001, 40.5% in 2002, and 42.7% in 2003). Comparison of the publication rate for 2000 with the combined rate for 2001–2003 showed that the increase was significant (OR 1.626; 95% CI, 1.190–2.224; P = 0.002). Comparison of publication rates between podium and poster presentations showed a significantly higher publication rate for podium presentations (OR 2.062; 95% CI, 1.547–2.749; P < 0.001).

The average time between congress presentation and publication was 17.7 months (±15.7 months; range −36 to 60 months). By year, the average times were 16.2 months (±16.5 months; range: −36 to 60 months) in 2000; 18.3 months (±17.0 months; range: −30 to 59 months) in 2001; 14.4 months (±14.4 months; range: −9 to 57 months) in 2002; and 21.2 months (±14.2 months; range: −2 to 56 months) in 2003. Twenty-five abstracts (3.0% of all 839 abstracts, 7.9% of all 317 publications) were published in peer-reviewed journals before congress presentation. These 25 abstracts were published on average 6.5 months (±8.5 months; range: 36–1 month) before the congress. With regard to the abstract submission deadline of each congress, a total of 8 abstracts were published before this deadline (on average 8.6 months ± 11.6 months; range: 0–30 months) and 17 abstracts were published between this deadline and the congress on average 3.3 months after the deadline (±1.5 months; range: 1–5 months).The publication rates over time are shown in Fig. 1.

Fig. 1.

The publication rate for all 317 publications during a follow-up period of 5 years after each congress. Twenty-five abstracts (3.0% of all 839 abstracts, 7.9% of all 317 publications) were published before congress presentation

Publications were identified in 55 different peer-reviewed journals, with the following top 5 journals: Spine (121 publications, 38.2% of all publications); European Spine Journal (73 publications, 23.0% of all publications); Journal of Spinal Disorders and Techniques (16 publications, 5.0% of all publications); Journal of Bone and Joint Surgery (British Volume) (10 publications, 3.2% of all publications); and Journal of Neurosurgery (9 publications, 2.8% of all publications). The mean impact factor for the journals at the individual times of publication was 1.798 (±1.048, range 0.000–10.232).

Table 3 shows the different publication-rate results in greater detail depending on the type of study. The publication rate for experimental studies was significantly higher than that for clinical studies (OR 1.656; 95% CI, 1.227–2.234; P = 0.001). The publication rates for clinical studies, analyzed according to the LoE, show that studies with a higher LoE have much higher publication rates than those with lower LoEs. Comparison of studies with LoE 1 and 2 (combined) with LoE 3–5 also shows a significantly higher publication rate for the higher-quality studies (OR 1.658; 95% CI, 1.141–2.409; P = 0.008). Randomized studies had a significantly higher publication rate than nonrandomized studies (OR 2.319; 95% CI, 1.345–3.998; P = 0.002). Prospective studies showed a significantly higher publication rate than retrospective studies (OR 1.738; 95% CI, 1.219–2.477; P = 0.002). Studies with a significant main result also had a significantly higher publication rate than those with a clearly nonsignificant main result (OR 2.398; 95% CI, 1.770–3.247; P < 0.001). Multicenter studies had a significantly higher publication rate than single-center studies (OR 2.671; 95% CI, 1.267–5.630; P = 0.008). Biomechanical studies showed a significantly higher publication rate than nonbiomechanical studies (OR 2.257; 95% CI, 1.451–3.510; P < 0.001).

Table 3.

Characteristics of abstracts and publications

	Abstracts		Publications		Publication rate (relative to associated no. of abstracts) (%)
	n	% (100% = 839 abstracts)	n	% (100% = 317 publications)
Type of study
Experimental study	268	31.9	124	39.1	46.3
Clinical study	532	63.4	182	57.4	34.2
Epidemiological study	19	2.3	7	2.2	36.8
Review	6	0.7	2	0.6	33.3
Case report	14	1.7	2	0.6	14.3
Clinical studies
Therapeutic study	404	48.2	134	42.3	33.2
Prognostic study	52	6.2	17	5.4	32.7
Diagnostic study	73	8.7	29	9.1	39.7
Economic study	3	0.4	2	0.6	66.7
Level of evidence in clinical studies
Level I	50	6.0	22	6.9	44.0
Level II	129	15.4	53	16.7	41.1
Level III	77	9.2	33	10.4	42.9
Level IV	275	32.8	74	23.3	26.9
Level V	1	0.1	0	0	0.0
Randomized study	59	7.0	31	9.8	52.5
Prospective study	255	30.4	105	33.1	41.2
Retrospective study	296	35.3	85	26.8	28.7
Study with significant main result	270	32.2	140	44.2	51.9
Study with nonsignificant main result	513	61.1	159	50.2	31.0
Single-center study	502	59.8	165	52.1	32.9
Multicenter study	30	3.6	17	5.4	56.7
Biomechanical study	90	10.7	50	15.8	55.6

In 162 of the 317 publications (51.1%), the composition of the author group was identical with that of the corresponding congress abstracts. In 104 publications (32.8%), at least one coauthor had been added. In 19 publications (6.0%), at least one main author had been added, and in 32 publications (10.1%) at least one coauthor and main author had been added. In 85.5% of the publications, the first author of the congress abstract was the main author of the publication.

The sample size in 217 of the 317 publications (68.4%) was the same as in the corresponding congress abstract. In 59 publications (18.6%) it was larger than in the congress abstract, in 24 publications (7.6%) it was smaller, and in 17 publications (5.4%) the feature of sample size was not applicable. The main result and conclusions were identical between the congress abstracts and the publication in all 317 publications.

Discussion

The overall publication rate in this series was 37.8%. This is approximately comparable with that reported for the NASS, with a publication rate of 40.0%, and only slightly inferior to that of the SRS with 47.4% and the ISSLS with 45.4% [27]. The figure shows the high quality of the programs at the conferences held by the SSE and indirectly validates the selection process used.

At conferences, the best submitted abstracts become podium presentations, while those of relatively lower quality become poster presentations and those with the poorest quality are rejected. The reviewers for the conferences only have short submitted abstracts available in order to assess the studies–—a sometimes difficult process due to the lack of information resulting from the limited length of the abstracts. The present study showed that there is a significantly higher rate of publication for podium presentations (48.4%) in comparison with posters (31.3%). This may be interpreted in two ways: either this finding corresponds well with the judgments made by the conference reviewers in differentiating between high quality and therefore better to publish podium presentations and slightly less quality and therefore more difficult to publish poster presentations. Or the authors of poster presentations are less encouraged to publish their studies by “only” finding their study accepted as poster instead of podium presentation. The finding that podium presentations have higher publication rates in comparison with posters was also reported by Preston et al. [25], who investigated publication rates for the Orthopedic Trauma Association. They found publication rates of 67% for podium presentations but only 52% for poster presentations when examining conference years 1994–1998 with a follow-up period of 6–10 years, including a total of 329 podium presentations and 486 poster presentations; the mean time to publication was 24.8 months for podium presentations and 21.6 months for poster presentations.

The average time to publication of 17.7 months in the present study roughly corresponds with that reported in other investigations (see Table 1). It would of course be desirable to have a more rapid publication process. However, it needs to be borne in mind that most authors mainly work in the clinical field and have only limited time for research. In addition, the review process for peer-reviewed journals including one or two revisions takes several months.

The problem of previously published studies being presented at conferences is well known. In this series, 3.0% of all abstracts and 7.9% of all publications had been published before the congress, 8 abstracts (1.0% of all abstracts) even before the abstract submission deadline of the congress (on average 8.6 months). However, conference organizers try to avoid abstract submission after publication. Comparable investigations in the field of orthopedics show rates of between 2 and 19% for publications appearing before the congress [7, 8, 11, 18, 20, 22, 23, 26]. Recently, a new trend could be observed: Some groups now aim to publish their studies prior to presentation at conferences mainly to avoid plagiarism, a growing problem in the days of internet. If the only way to secure one’s work is prompt publication, those papers will be banned from conferences unless the societies’ rules for abstract submission change and consider this point in the future. This problem is one of the recent challenges for conference organizers and societies.

The fact that most of the publications appeared in the journals Spine and the European Spine Journal, and the mean impact factor of 1.798 for all publications, indicates the high quality of the congress presentations. However, the fact that very often program committee members, who select conference abstracts simultaneously review for the main spine journals, especially the European Spine Journal in this case, probably influences the publication rate. This issue cannot be controlled and analyzed by the used study setup.

The examination of factors that may possibly have an impact on publication rates showed that higher quality research (high LoE, randomized studies, prospective studies, multicenter studies, biomechanical studies) achieved greater publication rates than lower quality studies. Although high-quality studies are often more laborious, expensive and difficult to perform they are the most valuable to reliably answer complex research questions corresponding well to the higher publication rates. Quality rather than quantity is to be recommended. Interestingly, the effect of publication bias [15] is obviously also present in the spine studies investigated. Presentations that had a clearly mentioned significant result had higher publication rates than those with nonsignificant results. It should be emphasized that this is a trend that ought to be resisted. Studies with nonsignificant results are of the same importance as comparable studies with significant results. It seems more attractive to present results including significant effects, of course. From the scientific point of view, however, nonsignificant results should not be underrated merely because of the lack of significance.

The analysis of the consistency of congress abstracts and publications showed that in more than half of the publications, the composition of the author group was identical with that in the congress presentation—a clear sign of trustworthiness and quality. The fact that authors were added in the other publications might be explained by the additional work involved in preparing the manuscript or completing the study.

The finding that the sample sizes were identical with those of the congress abstract in 68.4% of the publications shows that most of the studies presented at the SSE Annual Congress had been completed at the time. Only 18.6% of the publications had larger sample sizes, implying that the congress presentations were of incomplete studies. Smaller sample sizes in publications only occurred in 7.6% of the cases.

Analyses such as this raise the important issue of what happens to research studies that are presented in abstract form at conferences, but remain unpublished. All in all, 62.1% of the studies presented were not published within 5 years. It may be speculated why. Possibly, many of the unpublished studies might not have passed the full review process for peer-reviewed journals due to poor research quality. It should be emphasized that the review process for conferences is based on the short, limited abstracts submitted and not on the full-text manuscripts that are reviewed for journal publication. This might mean that studies of poorer quality are able to pass the review process for a conference simply because the reviewers do not have an opportunity to identify the limitations of a study, due to the limited length of the abstract. And of course, it needs to be kept in mind that congress organizers are in the need to fill the program and sometimes need to accept studies of slightly lower quality simply because there are only a limited number of high-quality studies available. These interpretations would call into question the reliability of citing congress abstracts before the corresponding full-text publication following a full peer-review process. The present authors would recommend that citations of congress abstracts should be kept to a minimum in scientific publications, guidelines, and textbooks. Nevertheless, the present results and especially the 100% agreement of results and conclusions between conference abstracts and publications in all 317 publications justify the yearly publication of SSE conference abstracts in the European Spine Journal.

Another possible explanation for the high rate of unpublished studies might be that medical researchers have insufficient time to pursue the publication process. The SSE might be able to support research groups to crown the studies that they have presented with a full-text publication. This could help prevent many valuable studies from being lost to those who were not able to attend the congress. Scientific research deserves greater recognition in society.

The only comparable investigation published in the field of spine surgery is by Wang et al. [27], who studied publication rates in the NASS, the SRS, and the ISSLS (Table 1). However, the study included far fewer details in comparison with the present one.

Limitations of the present study include the fact that reasons for nonpublication were not investigated. In addition, keeping in mind that the SSE was only 2 to 5 years old as to the investigated period (2000–2003) and that the SSE has grown substantially since then, it may be possible that results changed and further investigations of the SSE conference will find altered outcomes. However, as pointed out above, we support the follow-up period of 5 years, similar to comparable studies in the literature. As disclosure, the authors want to point out that the first and last authors are members of the SSE, which might theoretically lead to a certain bias. However, the authors assure that they conducted the study as objective as possible.

Conclusions

Papers presented at the SSE Annual Congress achieved a publication rate of 37.8% within 5 years, with a mean time between the conference and publication of 17.7 months and an average impact factor of 1.798.

Podium presentations had higher publication rates than posters.

The large proportion of conference presentations that remained unpublished should motivate the SSE to help presenters finish full-text publications in peer-reviewed journals.

It is a matter for critical discussion whether scientific publications, textbooks, and guidelines should cite and include information from conference presentations before the relevant research studies have successfully passed the review process required for publication in a peer-reviewed journal as a full manuscript. Implementation of findings based on conference presentations in clinical routine patient care should be done very carefully, and it must be borne in mind that studies presented at conference have not passed the full-text peer-review process.