Publication rates of abstracts presented at the Association of Chiropractic Colleges Educational Conference/Research Agenda Conference from 2002 to 2008

Abstract

Objective

The purposes of this study were to investigate the overall publication rates of presentations at the Association of Chiropractic Colleges Educational Conference/Research Agenda Conference (ACC/RAC) meetings (2002–2008), differences in the publication rates of platform vs poster presentations, and the consistency of the meeting abstract compared to the full-length journal article.

Methods

Abstracts were obtained from proceedings published in the Journal of Chiropractic Education. Literature searches using PubMed and the Index to the Chiropractic Literature (ICL) were performed to locate peer-reviewed journal articles based upon those abstracts. Whether the article was based upon a poster or platform presentation, and the congruence of the information in the abstract and article were recorded.

Results

We identified 776 proceeding abstracts, 249 of which eventually were published between 2002 and 2012. The overall publication rate was 32.2%. A total of 42.7% of platform presentations eventually were published vs 20.3% of posters. Congruency showed that 43.2% had the same title as the meeting abstract, 59.7% had the same authorship, and 88.8% had the same methods.

Conclusion

Publication rates of abstracts from spine and orthopedic surgery national meetings range from 34% to 59%. The ACC/RAC meetings have similar publication rates. More platform than poster presentations reach full publication. The congruency of ACC/RAC abstracts to published articles is higher than national meetings in other fields.

INTRODUCTION

The fundamental goal of science is discovery of new information, which is followed closely by communicating that new information to others. Communication in science takes place in several standard ways, such as presentations at scientific meetings and publication in scientific journals.¹ To communicate a new discovery rapidly, scientific meetings allow presenters to share their information by poster or platform presentations. Meeting presentations address a fairly restricted audience (ie, those attending the meeting), although published abstracts often accompany these presentations in the form of conference proceedings. To allow the widest distribution of the data to the scientific community, the generally accepted practice is to follow the meeting presentation with publication in a peer-reviewed journal. However, not all presentations successfully achieve publication. The process of publishing a journal article is rigorous and not all manuscripts survive the peer review procedure associated with publication. Studies of success to publication help us better understand what happens to material presented at conferences, and have shown that the overall publication rate of submitted manuscripts across medical specialties ranges from 36% to 66%.²

The annual conference of the Association of Chiropractic Colleges/Research Agenda Conference (ACC/RAC) is considered to be the premier chiropractic education and research meeting in the world.³ The ACC, comprised of accredited chiropractic educational programs,⁴ held its first annual educational conference in 1994, which included the initial peer review process and platform presentations.^5,6 The RAC was first held in 1996 with funding from Health Resources and Services Administration (HRSA), and Bureau of Health Professions,⁵ and focused on developing a chiropractic research agenda and skills of researchers through workshops and panel presentations. However, no peer-reviewed/contributed presentations were held at RAC. With HRSA funding ending and many noting that these separate conferences drew the same people, the ACC educational conference and RAC merged in 2002. After this time, the ACC/RAC has drawn the largest cadre of chiropractic scholars; thus, is an appropriate choice when evaluating publication rates of presentations from chiropractic research conferences.

As far as we are aware, there are no published studies concerning the publication rates from chiropractic research meetings. Therefore, the purposes of this study were to determine the overall publication rates of presentations at the ACC/RAC meetings (2002–2008), if there were differences in the publication rates of platform vs poster presentations, and the consistency of the information in the meeting abstract compared to the full-length journal article.

METHODS

Comprehensive literature searches were performed for all abstracts of poster and platform presentations at the 2002 through 2008 ACC/RAC meetings. The year 2002 was chosen because this was the first year that the ACC and RAC met together in a single event. It has been shown that more than 90% of conference abstracts achieve publication within 4 years following the meeting at which the presentation was made.⁷ Since this study was conducted in the summer of 2012, 2008 was selected as the cut-off year for this study to allow the appropriate 4-year period of time for the publication process to occur. The meeting abstracts were obtained from the Journal of Chiropractic Education (JCE), where they are published annually as the conference proceedings.

Publication Rates

Each meeting abstract was searched using PubMed (Medline) and Index to Chiropractic Literature (ICL) by the first author (BB) and confirmed by the second author (CC). The search strategy included the following procedures: the names of abstract authors were searched first.² If multiple publications were found for one author, keywords from the title were added to the search to identify if the individual presented abstract was published as a full paper. A meeting abstract was considered published if the title of the paper, authorship, and information in the publication contained substantial similarities as determined by the authors of this study. The content of the meeting abstract was compared directly to the publication content. If the publication had a smaller or larger sample size, the corresponding abstract was considered published only if it had an identical hypothesis and methods. If a publication was published before the meeting date, the meeting abstract was considered published only if the title of the paper, authorship, and information in the publication abstract were identical to that of the meeting abstract. If the initial search did not disclose a publication, the search was expanded to include the last name and first initial of each author listed, and manually cross-referenced with the key words in the title until a match was found, or it was determined that there was no match. Results of the searches were recorded in a Microsoft Excel (Microsoft Corp, Redmond, WA) spreadsheet, including abstract author(s), title, journal, year of publication, if indexed in PubMed and/or ICL, and if any different authors were added or removed compared to the abstract.

Platform Vs Poster Presentations

Once published papers were identified, the meeting abstracts that had matches were categorized by presentation style (platform vs poster). These data were tallied from the Excel spreadsheets.

Congruency Between Meeting Abstracts and Published Papers

The congruency of information in meeting abstracts and published articles was determined by direct comparison of the abstract in the proceeding to the published paper. This included the title, order, and composition of author groups; sample sizes; and main results and conclusions. Title comparisons were done word to word. Author groups were examined with special emphasis on congruency between first author on the publication and presenting author of the abstract. Other alterations in author groups were categorized, including authors that appeared on the meeting abstract, but not on the publication; authors that appeared on the publication, but not on the meeting abstract; and differences in order of appearance of author names between the meeting abstract and the publication. If present, sample size in the meeting abstract was compared to the sample size in the publication and whether they were the same or one was larger than the other. The subject matter of the meeting abstracts that were published as articles was categorized as basic science/experimental, clinical, education, or other (which included, but were not limited to, such types as epidemiology, history, literature reviews, policy, and economic issues). All of these data were recorded in the Excel spreadsheets.

Statistics

Descriptive statistics, including means, frequency counts, and percentages, were calculated for each of the variables: publication rates, platform vs poster presentations, and congruency of authorship, title, and content.

RESULTS

There were 776 abstracts included in the ACC/RAC proceedings published in the JCE between 2002 and 2008, with an average of 111 abstracts per year (Table 1). From these abstracts, 249 articles were published (ie, publication rate of 32.2%, Fig. 1). There were 12 papers (4.8%) published before the meeting at which each was presented.

Table 1.

Overall Rates of Abstracts and Articles

Figure 1.

The overall percentage rates of publication of abstracts from the ACC/RAC meetings between the years 2002 and 2008.

At most of the meetings, there were more platform than poster presentations (Table 2). The averages of platform and poster presentations in the years that were studied were relatively similar: 57.1 and 53.7, respectively. Platform presentations were more than twice (42.7%) as likely to result in a published article compared to poster presentations (20.3%, Fig. 2).

Table 2.

Publication Rates From Platform and Poster Presentations

Figure 2.

Percentage rates of publication from platform and poster presentations from the ACC/RAC meetings between the years 2002 and 2008.

Publications were identified in 37 different peer-reviewed journals (Table 3). Of the meeting abstracts published, there were more basic science/experimental studies compared to clinical or educational studies (Fig. 3). The other articles included, but were not limited to, epidemiology, history, literature reviews, policy, and economic topics.

Table 3.

Journals Publishing Articles Emanating From Abstracts of the ACCRAC

Figure 3.

Percentage of abstract types published as articles.

The congruency of the articles and the meeting abstracts showed that 43.2% of the articles had the same title as the meeting abstract (Fig. 4). A majority (59.7%) of the articles had the same authorship compared to the meeting abstract. Of the articles with a modified authorship, a different first author occurred 9.1% of the time. Other author differences included additional author(s), fewer author(s), and changes in the order of appearance of the authors. Of the articles 88.8% had the same methods, including sample size, as the meeting abstract (221 of 249). Therefore, there were 28 articles in which there was a difference in the methods compared to the meeting abstract. Of these articles 25 (10.0%) had an expanded protocol in which the number of subjects was higher than in the meeting abstract. There were 2 articles (0.8%) that had a lower number of subjects compared to the meeting abstract. In none of these cases did these changes in sample sizes alter the conclusions of the article compared to those of the meeting abstract. There was only 1 article (0.4%) in which the data in the article led the authors to a conclusion that was contradictory to that in the meeting abstract.

Figure 4.

Congruency between abstracts and articles.

DISCUSSION

Publication Rates

The findings from this study show that the overall publication rate for abstracts presented at the ACC/RAC meetings between the years 2002 and 2008 was 32.2%. This is similar to the publication rates of abstracts from scientific meetings associated with various spine and orthopedic surgery national organizational meetings, which have rates ranging from 34% to 59.4%.^1–2,7–15 If dissemination of research results is important to the chiropractic profession, then analyses of abstract-to-publication rates can help us better understand how many conference presentations are disseminated as articles. However, these rates will not inform us about what happens to research studies that are presented at meetings in abstract form, but are never published as journal articles. In the present study, 67.8% of the studies were not published as journal articles within 4 years of the meeting in which they were presented.

There are several hypotheses about what may prevent a researcher who has presented at a conference from publishing the presentation as a journal article. Reasons may include lack of time to prepare a manuscript,¹⁶ the study still is ongoing, difficulty in relationships with coauthors, lack of resources, lack of support from educational institution, lack of knowledge about how to publish, and lack of motivation. Another likely barrier to publication is lack of time due to heavy teaching loads.¹⁷ All but one of the ACC chiropractic colleges and universities are private institutions; therefore, resources are limited. Since the chiropractic colleges are not primarily research institutions, teaching is the main duty of the faculty members. Thus, there is little to no assigned time for research or scholarly activities.

Although it seems reasonable that an accepted conference submission (possibly 2000 words in length) should not require an inordinately large amount of time to produce an amended manuscript for submission to a journal, lack of time to develop it into a publishable paper may be a considerable issue. As well, many faculty members are not trained in scientific writing and publication methods, and, therefore, may lack the necessary skills. At present, it is unknown what percentage of full-time chiropractic college faculty members have assigned research time or release time for research and publication, and how this may correlate with abstract-to-publication rates.

Abstract-to-journal publication metrics have been studied in other fields. In the field of orthopedics, rates of publications appearing before the meeting are reported as between 2 and 19%.¹ The 4.8% rate for ACC/RAC meetings is similar. Some authors may aim at publishing before presenting at a meeting to avoid “plagiarism,”¹ whereas others try to publish early to try to avoid idea theft or other types of misconduct.¹⁸ This may represent a consideration for conference organizers and, if this trend is accurate, we might expect to see an increase in publication rates before presentation. However, in the world of chiropractic research, competition may not be so concerning.

While there may have been year-to-year differences in the annual publication rates between 2002 and 2008, the rates appear steady. This is similar to data from the Pediatric Orthopaedic Society of North America between 2002 and 2006,² but different from those from the Annual Congress of the Spine Society of Europe, where the rate increased.¹ The data in our study may mean that the barriers to publication were the same from 2002 to 2008. However, since many factors are associated with publication rates, and we did not measure these factors, we can only hypothesize. As well, looking for publication trends over only several years may not be revealing, as we would need to look at trends over a longer time to identify if there truly was a trend or not.

Platforms and Poster Presentations

Platform presentations were twice as likely as poster presentations to result in a journal article. This is similar to findings from meetings of the Pediatric Orthopaedic Society of North America and Spine Society of Europe, in which information from platform presentations was approximately 1.5 times more likely compared to poster presentations to be published as a journal article.^1,2 This is not the same in all cases, for example the Annual Congress of the German Society of Orthopaedics and Trauma Surgery were similar for poster and platform presentations in 2003.⁷

There are many hypotheses regarding the differences between presentation format and publication rates. One hypothesis is that platform presentations may be more scientifically advanced in their objectives, methods, and results.^1,2,8,9,19 Some suggest that reviewers and conference chairs may differentiate between higher quality (ie, more publishable, higher level of evidence, randomized trials) platform presentations and slightly lower quality (ie, less publishable, lower levels of evidence, case reports) poster presentations.^1,20 Thus, a platform presentation may be more publishable for various reasons, including research grant funding, team of authors, high profile, and expectations. Thus, de facto, publishable quality may be identified before a study has been developed into a manuscript. Another theory is that authors of poster presentations may feel less encouraged to publish, since their studies were “only” accepted as a poster, and, thus, may be less motivated. However, correlation does not imply causation. Thus, one cannot say that acceptance as a poster presentation will result in reduced chances of publication. Without directly measuring the authors' motivations, decision-making processes, and types and quality of the research published, it is unknown what influences drive platform or poster presentation publication.

Some conferences have reported a difference between publication rates of subject study matter.^2,7 However, based upon the mission of the ACC/RAC, subject matter of study may be somewhat different for the ACC/RAC meetings compared to more specialized spine and orthopedic surgery meetings. At ACC/RAC meetings, the highest percentage of presentations for years 2002 to 2008 was basic science/experimental studies (nearly 40%), with approximately 30% being clinical science studies, 14% educational studies, and the remaining 16% other types of studies, including, but not limited to, epidemiology, history, literature reviews, policy, and economic issues. At meetings of the Spine Society of Europe, and German Society of Orthopaedics and Trauma Surgery, more than 50% of the articles published from presented abstracts were from clinical studies and approximately 40% of articles were from experimental studies.^2,7 No articles from educational studies were reported from these meetings. The ACC/RAC has many purposes, including development and dissemination of research in basic science, clinical, and educational research. Therefore, the ACC/RAC call for papers includes a broad set of focus areas.^21,22 Whereas other professions have separate conferences for basic science/experimental, clinical, or educational research, ACC/RAC includes all research subject matter. Therefore, it is not possible for us to compare the ACC/RAC ratios to those of other scientific meetings.

Congruency

Congruency has been hypothesized as a characteristic of studies that have higher levels of credibility and quality.¹ However, to our knowledge there is no current evidence to support these opinions. The preliminary nature of the conference submission is based upon the idea that minor inconsistencies commonly are corrected through the peer review process between the meeting and final publication in a journal.¹⁰ There is a small percentage in which the interpretation of the data in the journal article changes enough that it essentially contradicts the information in the meeting abstract.^10,11,23 Another hypothesis may be that preliminary data with a small sample size are presented at the meeting, but with a larger sample size, early trends in the data may be reversed.

Analysis of the congruency of abstracts in the ACC/RAC proceedings to published articles showed that 43.2% had exactly the same title, 59.7% had the same authorship, and only 9.1% of the articles had an alternate first author. These numbers are similar to orthopedic meetings showing identical author composition between 60.2 and 83.9%, and changes in first author between 19 and 20%.⁷ There was a high congruency of protocols, including sample sizes, between the abstract and the published article (88.8%), which is similar to rates of other meetings (81.8%–66.3%).^1,10,11 Only 10% of the articles had a sample size bigger than that in the meeting abstract. This may mean that a majority of the studies presented at the ACC/RAC meetings were completed at the time of submission. There was only 1 instance in which the conclusions of the published article were contradictory to those of the meeting abstract, ie, 99.6% congruency. This is much smaller than the rates of inconsistencies in results between meeting abstracts and articles that were found in certain other studies. Bhandari et al¹¹ showed that 22% of articles in that study had a different result than the meeting abstract, and Kleweno et al¹⁰ showed a rate of incongruent results of 32.7%. On the other hand, in the study conducted by Schulte et al,¹ there was 100% agreement of results and conclusions between meeting abstracts and publications, and in another study led by Schulte et al,⁷ there was 99.5% congruence of conclusions between the meeting abstract and the publication.

The high rates of congruence in our study may be due to the relatively large maximum word count allowed in the submissions to the ACC/RAC meetings (ie, 1800–2000) and the comparatively rigorous peer review process applied to the ACC/RAC submissions. Through peer review and publication, changes typically are observed that include reporting of data, further refinement of wording of the title, and potentially bringing on additional authors (eg, statistician or other expert) who may help to strengthen the paper. Therefore, some differences in congruency are expected, as there is improvement from the initial time of presentation to publication.

Potential Effects of Conference Selection Process on Publication Rates and Congruency

It should not be assumed that the factors that determine publication in a journal are the same factors that determine selection for a conference presentation and designation of platform or poster format. Some conferences focus decisions for conference presentation on conference themes or timeliness of a topic; thus, some presentations are important, but may never have been intended to be published. For those factors that are aligned for publication and conference presentation, one may consider the scientific rigor of the study and if the information contributes new knowledge to the literature. Peng et al²⁴ demonstrated that presentations with “statistical analysis and number of authors to be the most significant predictive factors of publication success.” As well, it has been noted that studies with higher levels of evidence were more likely to be published.²⁵

Some meetings have short submissions (eg, less than 500 words), whereas the ACC/RAC allows authors up to 2000 words. The ACC/RAC Peer Review Board has set goals for the scientific presentations at the conference (Fig. 5), which includes increasing publication of presentations. In the early years of the ACC/RAC, only short submissions were reviewed for consideration; however, through feedback from the authors and the peer reviewers, a greater word count was requested. The reasons behind this were several-fold: (1) reviewers requested more content to judge the quality of the work, (2) there were complaints that authors could hide an incomplete or poor study with fewer words, and (3) more words allowed better demonstration of quality of the study. The feedback from the authors included that fewer words did not allow them to express their research and that those who had developed their work further felt that they possibly were more likely to bring their manuscript to completion and publication. Therefore, the submissions process was developed to include 2 items: a short abstract published in the conference proceedings (if accepted) and the longer blinded submission. For 2002 to 2008, the abstract word count ranged from 300 to 600 words. The blinded submission word limits ranged from 1800 to 2000 and could include references or tables/figures.²⁶ It was hoped that a higher word limit would assist peer reviewers with better reviews and to facilitate presentations to be published eventually. As the authors possibly would have produced 2000 words, it would seem easier for them to develop that work further into a manuscript for journal submission. For this study, we did not measure which of the published studies had blinded submissions with longer word counts, so we were not able to tell if the expanded word count correlated with publication rate.

Figure 5.

The long range goals of the ACC Peer-Review Committee.

Another possible reason for nonpublication may be that the manuscript did not pass the journal peer review process for it to go to full publication.^1,17 The ACC/RAC peer review process is solely for the purpose of selecting which presentations will be presented. Although peer reviewer comments are sent to the authors, the conference review process does not allow for revision, and acceptance of the submission does not depend upon whether the authors have addressed the concerns and comments from the reviewers. Because there is no revision process, it is likely that the presentation is different in quality (hopefully better, but potentially could be worse) than the original document that was submitted. After the conference, a submission that was accepted may not be developed fully into a manuscript and may not be ready for publication. If the author chose not to develop the manuscript further, it may not survive the peer review process and, thus, not succeed in being published. Another important consideration is the accept/reject rates of journals. A study of acceptance rates at major biomedical journals showed that acceptance for those journals was rare (ie, 6% accepted, 70% rejected outright, and 24% rejected after peer review).²⁰ Therefore, when considering publication rates for conference abstracts, 100% publication is not a realistic goal. It is possible that not all manuscripts submitted to a conference are fit for publication or should be published.

As an incentive for authors to transition from presentation to full publication, an awards competition was initiated by the conference organizers beginning with the 2007 ACC/RAC meeting.^27,28 The National Board of Chiropractic Examiners provided funding to the awards for best research papers in 2 categories: basic/clinic sciences and education. For the authors to receive the cash prize, the studies needed to be submitted successfully, pass through the peer review process, and be published in the Journal of Manipulative and Physiological Therapeutics or the Journal of Chiropractic Education, depending upon the subject matter of the research. Even though the publication rate of 2008 was not noticeably higher, the impact that this competition had on publication rates is difficult to elucidate. There are only a few prizes annually, so it would not be expected that they would have a significant influence on the overall publication rate.

Limitations

This study has several limitations. The search for articles was limited to the use of PubMed and ICL. Other databases, such as MANTIS, CINAHL, SCOPUS, were not used and, therefore, publications that appeared in journals that may be listed only in other databases would not have been identified. This could have led to an underestimation of the rates of publication. However, most chiropractic content may be found in the PubMed and ICL search engines. Also, 2008 was selected as the cut-off year to allow at least 4 years after the last meeting for subsequent publications to appear. It is possible for studies to be published 5 years (or more) after a meeting in which they were presented, which could lead to a further underestimation of the rate of publication. Another limitation is that we did not identify the time between the meeting presentation and the time of publication. As well, we did not contact or survey the authors for reasons why their studies were not published as full journal articles. It is possible that the manuscript was submitted and rejected, that it currently was in revision or in press for publication, or that the authors never submitted or intended to submit their research for publication. As well, we did not measure the levels of evidence or quality of research of the published papers. We also did not evaluate other factors associated with numbers of acceptances or designations of platform or poster sessions, which includes size of the facility, costs of poster boards, time available in the schedule, and number and quality of submissions for any given year.

CONCLUSION

The publication rates of ACC/RAC abstracts for years 2002 to 2008 compares favorably with the rates for other spine and orthopedic surgery national organizational meetings. Meeting abstracts from platform presentations were more than twice as likely to be published compared to those of posters. There was a high congruency rate of information between the meeting abstracts and articles.

CONFLICTS OF INTEREST

Claire Johnson is the peer review chair for the ACC/RAC. There were no funding sources for this study.