Publication rate of abstracts presented at the Australian Orthopaedic Association Annual Scientific Meeting

Abstract

Background

Presentations at scientific conferences are an important method of research dissemination, with abstracts often used to inform clinical practice. Abstract to publication ratio is a commonly used tool for determining meeting quality. The aim of this study was to determine the publication rate for abstracts presented at the Australian Orthopaedic Association Annual Scientific Meeting (AOA ASM) between 2012 and 2015 inclusive and identify reasons for non‐publication.

Methods

MEDLINE, PubMed and Google Scholar were searched to determine whether each abstract presented at AOA ASMs between 2012 and 2015 was associated with a full text publication in a peer‐reviewed journal. Where a publication could not be located, the presenter was contacted to confirm the reason for non‐publication.

Results

A total of 1130 abstracts were submitted (951 oral and 179 posters), and 573 resulted in full‐text peer‐reviewed publications (51%). The majority of publications (73%) were published within 2 years of presentation. There was no difference in likelihood of publication for oral presentations compared to posters, nor in the rate of publication across the 4 years of meetings. Common reasons for non‐publication were lack of time (32%), publication considered low priority (27%) and journal rejections (22%).

Conclusion

The overall publication rate for abstracts presented at the AOA ASM is 51%, which is an increase from the 1998 ASM (31%). This publication rate is higher than many similar Australian meetings and on par with other international orthopaedic and subspecialty meetings. Future research should investigate potential publication bias and methods to minimise barriers to publication.

Introduction

Presentation of research at national and international conferences is an important method of dissemination of research findings to relevant scientific communities. Research findings presented at scientific conferences are often used to inform clinical practice and guide future research. ¹ This can be problematic as many abstracts are based on preliminary results, contain insufficient details of the study methods, and have not been subjected to the more rigorous peer review processes required for full‐text publication in scientific journals. ² , ³ It is therefore important to evaluate the reliability and validity of the research presented at scientific conferences. One commonly used method to do this is to establish the number of conference abstracts that are subsequently published as full‐text articles in a peer reviewed journal. ⁴ For Australian medical scientific meetings, publication rates for presented research have been reported between 26% and 41%. ⁵ , ⁶ , ⁷ , ⁸ , ⁹ For international orthopaedic and orthopaedic subspecialty meetings, publication rates for presented research are between 24% and 67%. ¹⁰ , ¹¹ , ¹² , ¹³ , ¹⁴ , ¹⁵ , ¹⁶ , ¹⁷ , ¹⁸ , ¹⁹ , ²⁰ , ²¹ , ²² , ²³ , ²⁴ , ²⁵ , ²⁶ , ²⁷ , ²⁸ , ²⁹ , ³⁰ , ³¹

The Australian Orthopaedic Association Annual Scientific Meeting (AOA ASM) is one of the key scientific meetings for orthopaedic surgery and musculoskeletal health in Australia. Held annually in October each year, the meeting consistently attracts over 1000 delegates from Australia and across the world. A 2006 study ³² reported the publication rate of abstracts presented to the 1998 AOA ASM was 31%. No subsequent analysis of the publication rate for this meeting has been undertaken, and there has been no investigation of the reasons behind non‐publication of abstracts. The aim of this study was therefore to determine the publication rate for abstracts presented at the AOA ASM meetings between 2012 and 2015 inclusive, and identify the reasons for non‐publication. We also sought to determine the time between presentation and publication, common journals for publication, and impact of presenter job seniority and gender.

Methods

Search

A retrospective audit of all oral and poster abstracts presented at AOA ASMs between 2012 and 2015 was conducted using AOA abstract submission records and published meeting programs. Invited speakers and symposium presenters were excluded. The abstract title, specialty area, presentation mode, year of presentation, author and presenter details (job seniority, gender, time of presentation and affiliated institutions) were recorded.

For each abstract, the MEDLINE, PubMed and Google Scholar databases were searched to determine whether the abstract was associated with a full text publication in a peer‐reviewed journal. The search included all publications up to February 2020, which allowed a minimum period of 4 years for publication after presentation. ²⁷ First, the full title of each abstract was searched. If no match was located, a search by author name and keywords from the abstract title was done. If an associated article was still not located, the first author was searched and the author's list of publications were manually scanned for possible matches. A match was recorded as being found where substantial similarities between the presented abstract and the published paper were noted in relation to the title, authors and content (such as the patient cohort and study design). Where the abstract presented preliminary results of the published study, the publication was included. If the abstract included a full list of authors, the author list was checked against potential publications and if substantial differences were noted, the publication was excluded. If no corresponding publication was found after these steps, the abstract was noted as potentially unpublished. Publication of the AOA ASM abstract only in a journal supplement was not counted as a full‐text publication.

For abstracts where a corresponding publication was not located, the presenter was contacted via email inviting them to take part in a five‐question online survey requesting information on (i) whether the research covered in the abstract had been published in a peer‐reviewed journal prior to or since presentation, and (ii) the reason(s) for non‐publication if applicable (Supporting Information S1).

Analysis

Time to publication was calculated based on the month and year of presentation and the month and year in which the corresponding article was first published either online or in print, whichever was earliest. Articles published prior to the conference presentation were analysed separately to those published after presentation. Categorical variables were compared using the Chi‐square test, with differences considered significant at p ≤ 0.05. Data analysis was conducted in Microsoft Excel.

Ethics

Ethical approval for this study was granted by the University of Wollongong Human Research Ethics Committee (HREC2020/137).

Results

Table 1 shows that a total of 1130 abstracts (951 oral and 179 posters) were presented at the AOA ASM between 2012 and 2015. Abstracts were predominantly presented by men, with 126 (11%) presented by women. A total of 573 abstracts were subsequently converted to peer‐reviewed journal publications, resulting in an overall abstract to publication rate (APR) of 51% (Table 1). The majority of publications (n = 416, 73%) occurred within 2 years of presentation. Ninety‐nine abstracts (8.8%) were published prior to conference presentation (median 8 months prior). There was no significant difference in the rate of publication of oral and poster presentations, X ² (1, N = 1130) = 0.04, p = 0.84, or rate of publication across the 4 years of meetings, X ² (3, N = 1130) = 3.52, p = 0.32.

Table 1.

Publication rate of abstracts presented to AOA ASM by location and year

Year	Location	Abstracts presented n (%)			Converted to full paper publications n (%)
		Oral	Poster	TOTAL	Oral	Poster	TOTAL
2012	Sydney	238	39	277	125 (53%)	25 (64%)	150 (54%)
2013	Darwin	189	45	234	98 (52%)	25 (55%)	123 (53%)
2014	Melbourne	253	38	291	128 (51%)	18 (47%)	146 (50%)
2015	Brisbane	271	57	328	130 (48%)	24 (42%)	154 (47%)

Articles were published across 157 different journals (see Supporting Information Table S1 for a table of journals with five or more articles based on AOA ASM abstracts). The most common journal was the ANZ Journal of Surgery (44 publications) followed by the Journal of Arthroplasty (n = 41) and the Journal of Bone and Joint Surgery (n = 26). The median impact factor across all journals was 2.35.

A total of 472 articles were published after presentation and 99 articles were published prior to presentation. The median time between presentation and publication was 16 months for articles published after presentation (M = 20.39; SD 17.32). A greater percentage of abstracts presented by women were published (73/126, 58%) compared to those presented by males (500/1004, 50%) but the difference did not reach significance, X ² (1, N = 1130) = 2.96, p = 0.09. The percentage of female presenters who were non‐surgeons (38%) greatly exceeded that of male presenters (9%).

A chi‐square test showed a significant association between presenter role and publication, X ² (6, N = 1130) = 14.81, p = 0.02. Consultant surgeon was the most common presenter role (n = 484) with a publication rate of 53%. Research presented by orthopaedic trainees was published less frequently (43%) than research presented by interns (67%), medical students (59%) and unaccredited registrars (46%). Non‐surgeon presenters had a publication rate of 52%. See Supporting Information Table S2 for a breakdown of presenter roles.

Table 2 shows publication rate by subspecialty. Among the orthopaedic subspecialties, hip reconstruction had the highest rate of publication (65%) and paediatrics had the lowest (39%). The difference between specialty publication rates was not significant, X ² (10, N = 1105) = 11.54, p = 0.32. Medico‐legal and Orthopaedic Outreach also had low publication rates (20% and 0%, respectively), but were excluded from the chi‐square test as they are not orthopaedic sub‐specialties. Five presentations with an unknown specialty were also excluded.

Table 2.

Publication of abstracts presented to AOA ASM 2012–2015 by subspecialty area

Specialty	Number	Published n (%)
Arthroplasty	401	196 (49%)
Foot & Ankle	49	22 (45%)
General	143	75 (52%)
Hand	37	19 (51%)
Hip Reconstruction	26	17 (65%)
Paediatrics	51	20 (39%)
Shoulder & Elbow	97	55 (57%)
Spine	49	29 (59%)
Sports Knee	91	53 (58%)
Trauma	132	68 (51%)
Tumour	29	17 (59%)
Unknown	5	1 (20%)

Abstracts delivered by presenters from South Australia (64/110, 58%), Victoria (141/258, 55%), New South Wales (190/353, 54%) and Western Australia (39/83, 47%) had the highest APRs, while presenters from Queensland (62/153, 41%), the Australian Capital Territory (9/22, 41%), Tasmania (3/12, 25%) and the Northern Territory (0/4, 0%) had the lowest. Abstracts delivered by presenters from outside of Australia had a publication rate of 49% (64/130). Five abstracts had no state specified, one of which was published. A chi‐square test showed a significant association between presenter state and publication, X ² (8, N = 1125) = 20.48, p = 0.009.

Presenters can specify up to four institutions for their research on the abstract submission form. Most of the institutions with high numbers of publications (20+) are universities or teaching hospitals. A graph of the institutions with the highest number of publications is provided in Supporting Information Figure S1.

Seventy‐seven completed survey responses were received in response to 331 emails distributed to presenters with potentially unpublished abstracts (23.3% response rate). Seventeen respondents provided citation information for one or more publications, and 60 provided one or more reasons for non‐publication. The most common reasons for non‐publication were lack of time (32%), that publication was considered to be a low priority (27.5%), or that the articles were rejected (22%). Other reasons included insufficiently novel results (14.5%), articles were currently submitted and awaiting outcome (14.5%), lack of funding (7%), methodological limitations (7%) and similar results already being published (6%).

Discussion

The subsequent publication rate of conference abstracts in peer‐reviewed journals is an important method of evaluating the quality of the research presented at scientific meetings. ⁴ We found that just over half (51%) of abstracts presented at the AOA ASM between 2012 and 2015 were published as a full text article in peer‐reviewed journals. This is a significant increase from the publication rate of 31% reported for the 1998 meeting. ³² This may be the result of an increase in the number of scientific journals available to researchers ³³ or a greater emphasis having been placed on publishing within the profession.

The overall publication rate is higher than other Australian scientific meetings ³ , ⁴ , ⁵ , ⁶ , ⁷ and on par with other international orthopaedic and subspecialty meetings. ⁸ , ⁹ , ¹⁰ , ¹¹ , ¹² , ¹³ , ¹⁴ , ¹⁵ , ¹⁶ , ¹⁷ , ¹⁸ , ¹⁹ , ²⁰ , ²¹ , ²² , ²³ , ²⁴ , ²⁵ , ²⁶ , ²⁷ , ²⁸ , ²⁹ This is a reflection of the quality of the research being presented at this meeting, as high quality abstracts are more likely to be converted to peer‐reviewed publication. The quality of research is also supported by the high median impact factor of journals in which presented papers were published (2.35). While previous studies commonly report a higher APR for oral presentations compared to posters, ¹³ , ¹⁷ , ²⁷ we did not find a significant difference between publication rates for oral presentations compared to poster presentations at the AOA ASM. This suggests that the quality of the abstracts being presented as posters at the AOA ASM is equally as high as those selected for oral presentation. It is also possible that oral presentations are selected based on the overall focus of the meeting, or as a current ‘hot topic’ rather than on the basis of methodological strength. ³⁴ In relation to specialty area, we found particularly low abstract to publication rates for some specialties such as paediatric orthopaedics. We suggest that this may be related to perceived low value of the national AOA ASM among certain subspecialty groups when compared to subspecialty‐specific meetings that can provide a more relevant audience.

The low number of female presenters reflects the small percentage (4.5%) of female orthopaedic surgeons in Australia ³⁵ and is consistent with general trends in medical research, where women are less likely to publish, receive funding and be represented on journal editorial boards. ³⁶ , ³⁷ Recent changes to AOA's conference format, such as the purposeful inclusion of female panellists and free childcare and breastfeeding facilities, may have a positive impact on the percentage of female presenters. The numbers of female applicants to the orthopaedic training program are slowly increasing (18% for the 2022 intake) and this may also have an impact. These factors should be explored in future research.

We also noted that no publications resulted from abstracts presented in the Orthopaedic Outreach section. Orthopaedic Outreach is a charitable arm of AOA that provides surgical training and services to under‐developed Pacific Island nations. Publications from developed Western nations are frequently irrelevant to the practice of orthopaedic surgery in these regions. We therefore believe that it would be of benefit to encourage and support publication of articles that could help inform practice in regions supported by Outreach. Options may include establishment of a research mentorship program for Outreach presenters and promotion of reduced open access publication fees offered by orthopaedic journals.

In relation to trainee research output, the percentage of trainee presentations subsequently published was greatly exceeded by those of pre‐vocational doctors and medical students. It is likely that the demands of the training program make it more difficult for trainees to allocate time to conducting and publishing research. Furthermore, the research requirements of the orthopaedic Surgical Education and Training (SET) program could be met through presentation alone, leaving little incentive for trainees to publish their research. This is supported by an analysis of Australian orthopaedic research which reported that only 5% of publications authored by a trainee are published without a consultant surgeon as co‐author, and even less involve more than one trainee. ³⁸ This highlights the importance of senior surgeon support for trainees wishing to conduct research during the training program, particularly for first‐author publications. Senior surgeons should be encouraged to create opportunities for trainees to take the lead on research projects and provide support and resources to encourage presentation and publication of trainee‐led research. Pre‐vocational registrars and interns have additional motivation to publish, as a first‐author publication attracts a greater number of selections ‘points’ on an application for selection to the orthopaedic training program compared to presentation of the same research. The new AOA 21 training program, which replaced SET from the 2017 intake, places greater emphasis on publication, which may increase the trainee publication rate in future.

The most common reasons for non‐publication were lack of time and low priority for publication, which is consistent with previous studies. ³⁹ , ⁴⁰ Our results show that publications from abstracts are less frequent in institutions that are not part of a collaborative research network with a university or research institute. Additional funding for research administration support and protected time for research in smaller institutions may assist clinicians to find time to publish their findings. Medical colleges and associations could also prioritize the development and funding of resources and grants to support member research activities in smaller institutions without access to large grants and university funding.

This study has a number of limitations. The low survey response rate and limited number of databases that were searched may have caused some published studies to be overlooked. However, the databases chosen are the primary databases for medical and health topics and it is therefore unlikely that this had any significant impact on results. In addition, our survey response rate is not unusual compared to other similar studies. ⁴¹ , ⁴² It is possible that currently unpublished research may be published after the completion of this study, though our minimum follow‐up period of 4.5 years was longer than many similar studies. ⁴ , ¹⁵ , ¹⁹ , ²² , ²⁴ Other factors that may impact on publication such as study type, number of authors, positivity and significance of results were not investigated in this study and should be explored in future research.

Conclusion

The overall APR for the national AOA ASM was 51%, which is higher than many other Australian medical meetings and consistent with other national and international orthopaedic meetings. Publication most commonly occurred within two years of presentation and the most frequently used journals were the ANZ Journal of Surgery and the Journal of Arthroplasty. Common reasons for non‐publication included lack of time, low priority for publication and rejection by journals. Future research should investigate potential publication biases and methods to minimise barriers to publication, particularly support from senior colleagues and improved integration of research into orthopaedic training.

Author contributions

Belinda Balhatchet: Conceptualization; data curation; formal analysis; investigation; methodology; project administration; validation; visualization; writing – original draft; writing – review & editing. Heike Schütze: Conceptualization; formal analysis; methodology; supervision; validation; writing – review and editing. Anum Awais: Data curation; formal analysis; writing – review and editing. Nicole Williams: Conceptualization; data curation; formal analysis; methodology; supervision; writing – review and editing.

Conflict of interest

Belinda Balhatchet is an employee of the Australian Orthopaedic Association.

Supporting information

Appendix. Supporting information S1.

Table S1: Most common journals for publication of abstracts presented to AOA ASM 2012–2015.

Table S2: Publication of abstracts presented to AOA ASM 2012–2015 by presenter role.

Figure S3: Institutions with 20 or more published abstracts.