Publication Rates of Presentations at an Annual Meeting of the American Academy of Orthopaedic Surgeons

Abstract

Background

The Program Committee of the American Academy of Orthopaedic Surgery (AAOS) continually tries to improve the quality of the scientific program at AAOS meetings. However, according to the most recent study, the publication rate of papers presented at the 1996 annual meeting was only 34%.

Questions/purposes

To quantify the effects of these measures, we determined the 5-year publication rates in peer-reviewed journals of papers presented at the 2001 AAOS annual meeting.

Methods

Using the same methods described by Bhandari et al., we performed a comprehensive search of Medline and PubMed for subsequent publications of podium and poster presentations.

Results

The publication rates for all presentations were 49% at 5 years with poster and podium presentations at 47% and 52%, respectively. Among subspecialty divisions, the highest rate of publication was the sports medicine and arthroscopy category with 58% and the lowest was in the rehabilitative medicine category with 21%.

Conclusions

Less than 50% of abstracts presented at the 2001 AAOS annual meeting were published in the peer-reviewed literature at 5 years. As many studies presented will not pass the scrutiny of peer review, the information presented at the AAOS annual meeting should not be used as the sole guide to clinic practice.

Introduction

Presentations at national and international meetings provide an important forum to relay current research findings in all areas of medicine. Eventual publication of these presentations in a peer-reviewed journal is the desired outcome and would represent completion of a vigorous process that has been subject to extensive analysis and scrutiny of the data. Numerous reports across multiple medical subspecialties have documented that a substantial number of papers presented at these meetings never complete this process, with rates across various medical subspecialties ranging from 36% to 66% [1, 6, 8, 10, 14].

Orthopaedic surgery, like many other fields of medicine, is a continuously evolving field with new ideas and research being performed. The Program Committee of the AAOS is responsible for selection of papers to be presented at each annual meeting and has tried to improve the quality of the scientific program (Appendix 1). Although some authors have reported publication rates of AAOS meeting presentations as high as 44% [5, 8], in the most recent study on this topic Bhandari et al. reported a 5-year publication rate of only 34% [1]. These meetings comprise a substantial portion of continuing medical education, and the new knowledge acquired at these meetings often is rapidly incorporated into their clinical practice. For these reasons, it is important to question the scientific validity of these presentations through frequent assessment of their subsequent publication rates in the peer-reviewed literature, compare them with previously reported data, and analyze developing trends.

Our purposes therefore were to determine: (1) the overall publication rates of presentations from the 2001 AAOS meeting during a 5-year period; (2) whether there are differences in the publication rates of podium presentations versus posters during the same period of time; and (3) whether the publication rates of these presentations differ among various subspecialties.

Materials and Methods

Using the proceedings from the 2001 68th annual AAOS meeting, we performed a comprehensive literature search on all 756 listed presentations. The papers from the 2001 meeting were chosen to allow at least 5 years for subsequent publication and thus allow for comparison between prior published reports on this subject [12–14]. The presentations were subdivided into podium and poster presentations: 288 were podium presentations and 478 were poster presentations. We subsequently categorized each presentation into a specific subspecialty in orthopaedic surgery (Table 1). A few presentations were classified under more than one subspecialty. When these presentations were published, they were counted only once toward the overall rate of publication for the 2001 AAOS annual meeting. However, when reporting the rate of publication for the varying orthopaedic subspecialties, these abstracts were counted as having been a publication in each of those separate categories.

Table 1.

Podium and poster topics from 2001 annual AAOS meeting

Adult reconstruction—hip

Adult reconstruction—knee

Basic science

Foot and ankle

Hand and wrist

Musculotumor and metabolic disease

Pediatrics

Practice management/health policy

Rehabilitation medicine

Shoulder and elbow

Spine

Sports medicine/arthroscopy

Trauma

Miscellaneous

All of the papers listed in the AAOS proceedings were searched using Medline and PubMed using the methodology closest to the strategy used by Bhandari et al. [1]. Realizing the limitations, we chose to restrict our study to publications listed in these two databases because this approach has been used by others studying the same subject in orthopaedic surgery [1, 3, 5, 16] and other medical disciplines [4, 12, 13]. Each search included the names of the authors, listed presentation titles, and key words. The key words were defined as the main elements in the title and ranged from one key word to as many key words found in the title. A match was considered successful only if the title of the paper, authorship, and eventual abstract in the published manuscript contained substantial similarities as those published in the meeting proceedings. If the initial search did not reveal a publication, we then expanded the search to include the last name and first initial of every author listed and manually cross-referenced with the key words from the title until a match was found or it was determined that there was no match.

The time between presentation and publication was calculated for each abstract fitting these criteria. The time to publication was stratified into 1-year increments up to 5 years from the time of presentation. The increments were 1 year (March 2001 to March 2002), 2 years (April 2002 to March 2003), 3 years (April 2003 to March 2004), 4 years (April 2004 to March 2005), and 5 years (April 2005 to March 2006). Each eventual publication was classified according to time to publish and the subspecialty category for each abstract. A database of published presentations was created taking into account the total number of presentations, total poster versus total podium presentations, and each paper’s subspecialty category. Abstracts from this meeting published after 5 years were not considered in this study. Once publication rates were formulated, the data were compared with other data from previous studies of publication rates at earlier AAOS meetings using Pearson’s chi square test.

Results

From the 756 presentations at the 2001 68th annual AAOS meeting, we identified 367 published peer-reviewed papers by March 2006. This resulted in an overall publication rate of 49% (Table 2). The majority (63%) of the presentations were published within the first 2 years after being presented at the AAOS meeting (Fig. 1; Table 3). The total number of published presentations increased each year; however, the likelihood of publication decreased after the third year (Fig. 2).

Table 2.

Cumulative podium and poster presentation publication rates

Total	Publication Year 1	Publication Year 2	Publication Year 3	Publication Year 4	Publication Year 5	Not published
756	115 (15%)	228 (30%)	302 (40%)	333 (44%)	367 (49%)	389 (51%)

Fig. 1.

The publication rates of poster and podium abstracts after presentation at the 2001 AAOS Annual Meeting are shown. The majority of the papers eventually published in peer reviewed literature (63%) were published within 2 years after presentation.

Table 3.

Podium and poster presentation publication rates per year

Total	Publication Year 1	Publication Year 2	Publication Year 3	Publication Year 4	Publication Year 5	Not published
756	115 (15%)	113 (15%)	74 (10%)	31 (4%)	34 (5%)	389 (51%)

Fig. 2.

The cumulative publication rates versus the yearly publication rates of podium and poster presentations are shown. The total number of published papers increased yearly after the 2001 AAOS Meeting. However, the number of published papers and the likelihood of publication decreased 3 years after initial presentation.

The overall 5-year publication rates for podium and poster presentations were 52% and 47%, respectively (Tables 4, 5). The publication rates for the podium and poster presentations followed the general trend of the overall publication rates, but podium presentations had a publication rate of 52% at 5 years when compared with 47% for poster presentations. A podium presentation had the same likelihood (p = 0.168) of publication compared with a poster presentation (Table 6).

Table 4.

Cumulative podium presentation publication rates

Total	Publication Year 1	Publication Year 2	Publication Year 3	Publication Year 4	Publication Year 5	Not published
288	51 (18%)	95 (33%)	122 (42%)	134 (47%)	149 (52%)	139 (48%)

Table 5.

Cumulative poster presentation publication rates

Total	Publication Year 1	Publication Year 2	Publication Year 3	Publication Year 4	Publication Year 5	Not published
468	64 (14%)	133 (29%)	180 (38%)	199 (43%)	218 (47%)	250 (53%)

Table 6.

Podium versus poster presentations at 2001 AAOS meeting

Type of presentation	Published	Unpublished	Total	Odds ratio (confidence interval)	p Value
Podium	149 (52%)	139	288	1.229 (0.916–1.649)	0.168
Poster	218 (47%)	250	468

Among subspecialty publications, the highest rate of publication was in the sports medicine and arthroscopy category with 58% (n = 41) being published at 5 years (Table 7). Adult reconstruction of the hip had the highest number of presentations with 127 total abstracts. The publication rate for this category was 37%. The lowest overall rate of publication was among papers presented in the rehabilitative medicine category with 21% (n = 3) being published at 5 years.

Table 7.

Cumulative subspecialty podium and poster publication rates

Subspecialty	Total	Publication Year 1	Publication Year 2	Publication Year 3	Publication Year 4	Publication Year 5	Not published
Adult reconstructtion of the hip
	127	15 (12%)	27 (21%)	44 (35%)	46 (36%)	47 (37%)	80 (63%)
Adult reconstructtion of the knee
	94	15 (16%)	27 (29%)	34 (36%)	38 (40%)	43 (46%)	51 (54%)
Basic science
	52	8 (15%)	17 (33%)	19 (37%)	23 (44%)	25 (48%)	27 (52%)
Foot and ankle
	25	2 (8%)	8 (32%)	9 (36%)	11 (44%)	11 (44%)	14 (56%)
Wrist and hand
	32	6 (19%)	8 (25%)	8 (25%)	9 (28%)	9 (28%)	23 (72%)
Pediatrics
	49	9 (18%)	17 (35%)	19 (39%)	21 (43%)	25 (51%)	24 (49%)
Tumor
	31	6 (19%)	11 (35%)	15 (48%)	15 (48%)	17 (55%)	14 (45%)
Health policy
	31	7 (23%)	11 (35%)	15 (48%)	17 (55%)	17 (55%)	14 (45%)
Rehabilitative medicine
	14	2 (14%)	2 (14%)	2 (14%)	3 (21%)	3 (21%)	11 (79%)
Shoulder and elbow
	74	8 (11%)	24 (32%)	30 (41%)	33 (45%)	39 (53%)	35 (47%)
Spine
	72	19 (26%)	27 (38%)	34 (47%)	35 (49%)	41 (57%)	31 (43%)
Sports medicine
	71	9 (13%)	20 (28%)	34 (48%)	37 (52%)	41 (58%)	30 (42%)
Trauma
	108	15 (14%)	35 (32%)	47 (44%)	55 (51%)	60 (56%)	48 (44%)
Miscellaneous
	16	1 (6%)	6 (38%)	7 (44%)	7 (44%)	8 (50%)	8 (50%)

Discussion

The presentation of a paper at a national or international meeting is a well-accepted method of conveying the conclusions of research. The eventual publication of such data in a peer-reviewed journal represents a more permanent, more accessible, and validated culmination of the scientific process [6]. Although many abstracts presented at the AAOS annual meeting are cited in the orthopaedic literature and are used to guide orthopaedic practice, not all presentations undergo the rigorous process of peer review and the path to eventual publication. The purposes of our study therefore were to determine: (1) the overall publication rates of presentations at the 2001 AAOS meeting; (2) whether there are differences in the publication rates of the podium versus poster presentations during the same time; and (3) whether the publication rates of these presentations differ among various subspecialties.

This study had several limitations. First, we limited our search to articles found in PubMed and Medline and did not consider journals not indexed in PubMed and Medline. Our search did not include the EMBASE or Cochrane databases and, therefore, would have failed to identify any publications in peer-reviewed journals indexed only in either of these two databases. EMBASE is a large biomedical and pharmacologic database containing more than 12 million records from 1974 to the present compared with Medline and PubMed, which contains more than 15 and 18 million records dating to 1949 and 1966, respectively [7, 10]. Although several studies in various areas of biomedical research (eg, pharmacology and family medicine) suggest the use of EMBASE yielded more results compared with the use of Medline or PubMed [2, 9, 17], this may not translate to specialties like orthopaedics. Recently, Whitehouse et al. compared the use of Medline, Google scholar, and EMBASE regarding publication rates of orthopaedic hip abstracts presented at international meetings and found that Medline and Google scholar identified significantly more publications than EMBASE [16]. Furthermore, Slobogean et al. examined the searches of abstracts and studies used in orthopaedic meta-analyses among various databases including Medline, EMBASE, Cochrane, CINAHL, and Web of Science, and reported positive search rates of 90% and 81% when using Medline and EMBASE respectively [15]. Combining Medline and EMBASE increased the positive yield to 91%. Searching the Cochrane databases yielded only 36 titles not indexed in Medline or EMBASE. Slobogean et al. concluded that a high percentage of primary research in orthopaedics can be found using the major databases, and the use of additional databases was unlikely to increase the yield of published manuscripts [15]. Therefore, although we recognize that our study design may underestimate the eventual rate of publication, our methods are consistent with those used in previous studies on this topic in limiting searches to Medline and PubMed databases [3–5]. Second, we limited our literature search to a 5-year period after the AAOS meeting. Although we recognize that some studies could be published greater than 5 years after initial presentation, several studies have reported that greater than 90% of papers from a scientific meeting are published within 4 years of presentation [12–14]. In addition, there is a possibility that these late publications may contain additional data, followup, or different conclusions compared with the given presentations. Thus, because the number of these outliers is historically low, the overall publication rate likely would not be affected by their inclusion. Finally, because of the human element involved in these literature searches, comparison of our results with the publication rates from prior meetings is limited. We purposely designed our search criteria to match those used by Bhandari et al. [1] to allow us to compare our results. Although the basic search algorithm is essentially identical to the prior study, a match ultimately is determined by a person and not by computer alone. Because there is a degree of subjectivity built into each search, this may limit comparisons across studies because small changes in criteria can yield different results.

Our data suggest an overall publication rate of 49%. These data are consistent with published results. Hamlet et al. reported a publication rate of 46% of presentations from AAOS meetings from 1990 to 1992 [6]. Furthermore, Murrey et al. reported a similar publication rate of 44% for presentations from the 1993 AAOS meeting [10]. However, in the last study on this subject, Bhandari et al. reported a publication rate of only 34% from the 1996 meeting [1]. Although direct comparisons between our study and that of Bhandari et al. [1] must be made with caution owing to variations in application of the methods, presuming these rates are truly different, a potential reason may be the result of the AAOS program committee’s continual efforts to increase the scientific quality of the presented papers (Appendix 1). This effort may account for the higher publication rates of abstracts presented at subsequent meetings. However another explanation for the apparent increase in publications may be attributable to an increase in the number of journals and forums for publications. Thus, future studies of publication rates at more recent annual meetings are needed to quantify and chart the trend of these efforts. Nevertheless, our study shows the trend of publications in peer-reviewed journals after presentation at an AAOS meeting has remained relatively consistent (Tables 8, 9).

Table 8.

Publications from 2001 annual AAOS meeting compared with previous meetings

Meeting	Published	Unpublished	Total	Odds ratio (confidence interval)	p Value
2001 AAOS (current study)	367 (49%)	389	756
1996 AAOS Jasko et al. [8]	159 (34%)	306	465	1.816 (1.430–2.305)	< 0.0001
1993 AAOS Hamlet et al. [6]	251 (43%)	322	573	1.210 (0.973–1.505)	0.086
1992 AAOS Elder et al. [4]	201 (46%)	240	441	1.127 (0.890–1.425)	0.321
1991 AAOS Elder et al. [4]	228 (46%)	273	501	1.130 (0.901–1.416)	0.291
1990 AAOS Elder et al.[4]	239 (46%)	284	523	1.121 (0.897–1.402)	0.316

Table 9.

Similar published studies in orthopaedics

Study	Specialty meeting	Publication rates
Hamlet et al. [6]	AAOS 1990–1992	46%
Murrey et al. [10]	AAOS 1993	44%
Bhandari et al. [1]	AAOS 1996	34%

We found the overall publication rates for podium and poster presentations were 52% and 47%, respectively, at 5 years. The data suggest the quality of poster and podium presentations are equivalent, and this information can allow physicians at the AAOS meeting to view the information presented in either form as having a similar likelihood of eventually being published in a peer-reviewed journal.

Not all studies presented at AAOS meetings will pass the scrutiny of peer review and lead to publication. Before changing one’s practice strategies based on these presentations, surgeons should be aware the information may not pass peer review and may not be valid. To our knowledge, the impact of AAOS abstracts on practice patterns has not been studied. This topic and publication trends of more recent AAOS meetings can provide future direction for areas of research.

Appendix 1