Full-text publication rate of abstracts presented at the Japan Primary Care Association Annual Meetings (2010–2012): a retrospective observational study

Abstract

Objective

To determine the publication rate of abstracts presented at the Japan Primary Care Association Annual Meetings and the factors associated with publication.

Design

A retrospective observational study.

Participants

All abstracts presented at the Japan Primary Care Association Annual Meetings (2010–2012).

Main outcome measures

Publication rates were determined by searching the MEDLINE database for full-text articles published by September 2017. Data on presentation format (oral vs poster), affiliation of the first author, number of authors, number of involved institutions, journal of publication and publication date were abstracted.

Results

Of the 1003 abstracts evaluated, 38 (3.8%, 95% CI 2.6% to 5.0%) were subsequently published in peer-reviewed journals indexed in the MEDLINE database. The median time to publication was 15.5 months (IQR, 9.3–29.3 months). More than 95% of published abstracts were published within 4 years. The publications appeared in 23 different journals (21 English-language journals and two Japanese-language journals). Based on univariate analysis using binary logistic regression, publication was more frequent for oral presentations (7.3%vs2.0% for poster presentations; OR 3.91,95% CI 1.98 to 7.75), and for first authors affiliated with university-associated institutions (6.4%vs2.4% for first authors affiliated with non-university-associated institutions; OR 2.75,95% CI 1.42 to 5.30). Based on multivariate analysis, oral presentation and first author affiliation with a university-associated institution were still the only independent predictive factors for publication (adjusted OR 3.50(95% CI 1.72 to 7.12) and adjusted OR 2.35(95% CI 1.19 to 4.63), respectively). Even among 151 abstracts presented orally by first authors affiliated with a university-associated institution, only 18 abstracts (11.9%) were subsequently published in peer-reviewed journals.

Conclusions

The publication rate of abstracts presented at the Japan Primary Care Association Annual Meetings was extremely low. Further studies are warranted to investigate the barriers to publication among investigators who participate in conferences where the publication rate is extremely low.

Strengths and limitations of this study.

• This is the first study to evaluate the publication rate of abstracts presented at the Japan Primary Care Association Annual Meetings.

• The interobserver reliability of the publications corresponding to the presented abstracts was evaluated.

• Publication rates were determined by searching a single database.

• Authors were not contacted to elicit whether their abstracts were published in peer-reviewed journals.

Introduction

A scientific presentation at an annual meeting is considered an initial method to share novel research findings before their publication in a peer-reviewed journal.^1–3 However, a past systematic review reported that only 45% of abstracts presented at annual meetings are subsequently published.⁴ Furthermore, the annual meetings varied greatly in the publication rate of presented abstracts which ranged from 8% to 81%. Recent studies conducted from 2014 to 2016 also showed that the publication rate of abstracts presented at annual meetings ranged from 12% to 67%.⁵

Several factors associated with publication of abstracts presented at annual meetings have been evaluated. Past systematic reviews reported that positive factors associated with publication were abstracts showing positive results, oral presentation (vs poster presentation), randomised trial study design, higher quality of abstracts, country of the meeting and specialty of the meeting.^{4 6} However, even among abstracts describing results of randomised controlled trials (RCTs) with positive results, no more than 70% of them are subsequently published in a peer-reviewed journal.^{4 7–11} Furthermore, the publication rate of those abstracts ranges from 38% to 87% depending on the annual meeting.^{4 7–11} Therefore, unknown factors may affect the publication rate of abstracts presented at annual meetings.

Few studies have evaluated the publication rate of abstracts presented at Japanese medical specialty meetings.2 4–6 12 Furthermore, to our knowledge, no studies have investigated the publication rate of abstracts presented at the Japan Primary Care Association Annual Meetings (JPCAMs).^{4 6} Given that a recent study reported that the proportion of Japanese articles in five high-impact international primary care journals was only 0.15%,¹³ the publication rate of abstracts presented at JPCAMs may be extremely low. Furthermore, if the publication rate of abstracts presented at JPCAMs is much lower than that of abstracts presented at primary care meetings in other countries, this may give us some insights regarding the most influential factors associated with abstract publication. Therefore, we conducted this study to determine the publication rate of abstracts presented at JPCAMs and the factors associated with their publication.

Methods

Study design

A retrospective observational study of abstracts presented at JPCAMs (2010–2012) was conducted. The Japan Primary Care Association is one of the major Japanese medical academic organisations, to which more than 10 000 physicians belong.¹⁴ The aims of this study were to determine the proportion of abstracts presented at JPCAMs that were eventually published as full-text articles and to evaluate the factors associated with publication. These meetings were chosen to allow sufficient time to publication because more than 95% of published articles are published within 5 years of presentation.⁴ All oral and poster presentations were included. Duplicate abstracts within the same conference and retracted abstracts were excluded. Abstracts published before the conference were also excluded based on a past study.¹⁵ Institutional review board approval was not required for this study.

Outcome measures and search strategy

The primary outcome was subsequent publication of abstracts presented at JPCAMs. Based on a previous study,¹ publication was determined by searching full-text publications in the MEDLINE database. Only articles published from the time of the annual meeting to September 2017 were included.

For the search strategy, the name of the first author was used as a keyword to search for the article. If the initial search identified no publication corresponding to the presented abstract, the name of the second author was used to search. Abstracts were considered published if a matching full-length article was identified using this search strategy. Based on a previous study,¹ a brief report or research letter was also considered a published article because these types of publications are subject to peer review and are indexed in the MEDLINE database. Retrieved publications were compared with the corresponding abstracts to ensure that they represented the same work. Only published articles that were nearly identical in terms of targeted population, hypothesis and study design were judged to be the same work.⁴ Articles that included some of the data presented in the abstract (eg, a smaller cohort) were also regarded as the same work.¹⁶ Abstracts for which it was difficult to judge whether the identified article represented the same work were discussed, and the decision was made by consensus among the two authors of this report (JK and TY). Abstracts were considered unpublished if no matching full-length article was identified from this search strategy. Authors of the abstracts were not contacted to elicit whether the research had been published in a peer-reviewed journal.

We conducted a preliminary study as the first part of this study to evaluate the interobserver reliability for the outcome assessment. For the first 100 abstracts presented at the Japan Primary Care Association 2012 Annual Meeting, two investigators (JK and TY) independently searched for publications corresponding to the presented abstracts using two databases (MEDLINE and Google Scholar) based on the recommendation of a past systematic review to use at least two databases for a search.⁴ Furthermore, in the preliminary study, we used the name of the first, second and last authors as keywords in the search. In this preliminary study, the interobserver agreement was very good for the publication search (kappa 0.85).¹⁷ This finding was consistent with a past study showing good interobserver agreement for a publication search.¹⁸ Therefore, we divided the included abstracts and performed the search (JK for abstracts of poster presentations in 2011 and all abstracts in 2010; TY for abstracts of oral presentations in 2011 and all abstracts in 2012). In addition, all identified publications in the preliminary study were searchable without using Google Scholar or searching by the last author’s name as a keyword. Therefore, we used only the MEDLINE database and searched the first and second authors’ names as keywords in the present study.

Characteristics

For abstracts presented at JPCAMs, the following information was extracted: the year of the annual meeting, the presentation format, the affiliation of the first author, the number of authors involved and the number of involved institutions. For the identified articles corresponding to presented abstracts, the name of the journal and the publication date were retrieved.

Statistical analysis

Sample size was determined based on past studies. The three consecutive JPCAMs were chosen because the mean sample size of past studies was approximately 400 and because the mean number of conferences included in past studies was 3.8.^4–6 For the primary outcome, the total publication rate was calculated, and the 95% CI was determined. The primary outcome was also calculated according to the year of the annual meeting, presentation format, affiliation of the first author, number of authors involved and number of authors’ affiliations. For the published abstracts, the median time (months) from the annual meeting to publication was calculated. The accumulated number and proportion of published abstracts among all published abstracts every 6 months were determined. In addition, information on peer-reviewed journals in which the abstracts were published was presented by descriptive statistics. The associations between publication and the following variables were evaluated by univariate analysis using binary logistic regression: the year of the annual meeting (2010, 2011 or 2012), presentation format (oral or poster), affiliation of the first author (university-associated or non-university-associated institution),¹⁵ number of authors involved (n<3 or n≥3),¹⁵ the number of authors’ affiliations (single-centre authorship or multicentre authorship).¹⁹ Multivariate analysis was also conducted using these variables. RCT design was not included as a variable because few abstracts described results of RCTs. These analyses were conducted using Stata V.15 or the Bellcurve for Excel statistical software package V.2.11 (Social Survey Research Information, Tokyo, Japan), and the threshold for significance was p<0.05.

Patient involvement

No patients were involved in determining the research question or outcome measures, nor were they involved in developing plans to design or implement the study. No patients were asked for advice during the interpretation or writing of the results. There are no plans to disseminate the results of this research to the relevant patient community.

Results

All 1008 abstracts from these meetings were reviewed. Five abstracts were excluded for various reasons (one for withdrawal, one for duplicate abstracts within the same conference and three for publication before the conference). Thus, a total of 1003 abstracts (237 in 2010, 314 in 2011 and 452 in 2012) presented by 765 first authors were evaluated. Of 765 first authors, 176 (23.0%) presented two or more abstracts as a first author during three JPCAMs (2010–2012). Of 1003 abstracts, 343 (34.2%) were oral presentations, and 660 (65.8%) were poster presentations. Only two abstracts described results of RCTs (0.2%). The median number of authors was 5.0 (IQR 3.0–8.0). Of all abstracts, 38 (3.8%, 95% CI 2.6% to 5.0%) were published in a peer-reviewed journal indexed in the MEDLINE database. The median time to publication was 15.5 months (IQR 9.3–29.3 months). Approximately two-thirds of those published abstracts were published within 2 years, and more than 95% were published within 4 years (table 1).

Table 1.

Distribution of 38 published abstracts by time from presentation at a Japan Primary Care Association Annual Meeting (2010–2012)

Time from the annual meeting	Number of publications	Accumulated number of publications, n (%)
0–6 months	4	4 (10.5)
7–12 months	10	14 (36.8)
13–18 months	6	20 (24.1)
19–24 months	5	25 (65.8)
25–30 months	4	29 (76.3)
31–36 months	4	33 (86.8)
37–42 months	4	37 (97.4)
43–48 months	0	37 (97.4)
49 months or later	1	38 (100)

Values are the number of published abstracts, with the percentage of the total number of published abstracts in parentheses.

Table 2 shows the characteristics of the abstracts that were published. Based on univariate analysis using binary logistic regression, publication was more frequent for oral presentations (7.3% vs 2.0% for poster presentations; OR 3.91, 95% CI 1.98 to 7.75) and for first authors affiliated with university-associated institutions (6.4% vs 2.4% for first authors affiliated with non-university-associated institutions; OR 2.75, 95% CI 1.42 to 5.30). The number of authors involved, number of involved institutions and year of the conference were not associated with subsequent publication. Based on multivariate analysis, oral presentation and first author affiliation with a university-associated institution were the only independent predictive factors for publication (adjusted OR 3.50 (95% CI 1.72 to 7.12) and adjusted OR 2.35 (1.19 to 4.63), respectively). Of 151 abstracts presented as oral presentations by first authors affiliated with university-associated institutions, 18 abstracts (11.9%) were subsequently published in peer-reviewed journals.

Table 2.

Factors associated with publication among abstracts presented at the Japan Primary Care Association Annual Meetings (2010–2012)

	Total, n	Number (%) of publications*	Univariate analysis		Multivariate analysis†
			OR (95% CI)	P values	OR (95% CI)	P values
Year of the conference
2010	237	10 (4.2)	1 (reference)
2011	314	10 (3.2)	0.75 (0.31 to 1.82)	0.52	0.98 (0.39 to 2.48)	0.97
2012	452	18 (4.0)	0.94 (0.42 to 2.07)	0.88	1.35 (0.59 to 3.08)	0.47
Presentation format
Poster	660	13 (2.0)	1 (reference)	<0.001	1 (reference)	<0.001
Oral	343	25 (7.3)	3.91 (1.98 to 7.75)		3.50 (1.72 to 7.12)
Number of authors’ affiliations (multicentre authorship)
Single-centre	512	14 (2.7)	1 (reference)	0.07	1 (reference)	0.11
Multicentre	491	24 (4.9)	1.83 (0.93 to 3.58)		1.76 (0.88 to 3.55)
Number of authors
<3	198	4 (2.0)	1 (reference)	0.15	1 (reference)	0.57
≥3	805	34 (4.2)	2.14 (0.75 to 6.10)		1.38 (0.46 to 4.11)
Affiliation of the first author
Non-university-associated institution	659	16 (2.4)	1 (reference)	0.002	1 (reference)	0.01
University-associated institution	344	22 (6.4)	2.75 (1.42 to 5.30)		2.35 (1.19 to 4.63)

*Values are the number of published abstracts, with the percentage of the total number of published abstracts according to subgroup classified by each variable in parentheses.

†The following variables were used: year of the conference, presentation format, number of authors’ affiliations, number of authors and affiliation of the first author.

Table 3 shows the journals in which the abstracts were published. In total, 38 abstracts presented at the 2010, 2011 and 2012 JPCAMs were published in 23 different journals (21 English-language journals and two Japanese-language journals). Of those, 36 abstracts (94.7%) were published in English-language journals.

Table 3.

List of journals in which the abstracts presented at the Japan Primary Care Association Annual Meetings (2010–2012) were published

Journal	Number of publications	Percentage of publications
Asia Pac Fam Med	5	13.2
Intern Med	5	13.2
Int J Gen Med	4	10.5
BMC Fam Pract	3	3.6
Geriatr Gerontol Int	2	2.4
Patient Prefer Adherence	2	2.4
Others*	17	44.7

*These consisted of 17 journals (two Japanese-language journals and 15 English-language journals).

Discussion

This study showed that the publication rate of abstracts presented at JPCAMs was only 3.8%. Our findings support the low proportion of Japanese articles in the five high-impact international primary care journals.¹³ Given that the publication rate of abstracts presented at annual scientific meetings for primary care in other countries is more than 40%,1 4 5 15 the publication rate of abstracts presented at JPCAMs is extremely low. Furthermore, to our knowledge, the abstract publication rate observed in the present study was the lowest among all studies conducted in past decades, regardless of specialty.^4–6

Several explanations for this result are considered. First, language could be a barrier to publication of articles in international peer-reviewed journals for Japanese investigators.²⁰ Past studies reported that author origin (from non-English-speaking countries vs from English-speaking countries) affected the abstract publication rate.^{4 6 21} However, this reason is unlikely to account for our results because the publication rate of abstracts presented at the Annual Research Meeting of the Japanese Orthopaedic Association is similar to that of orthopaedic meetings in other countries.2 4 5 22 23 Therefore, it appears that this low publication rate is specific to JPCAM regardless of country or specialty. Second, the extremely low proportion of abstracts describing results of RCTs might have affected the publication rate of abstracts. However, past studies reported that approximately 40% of presented abstracts describing results of non-RCT research were subsequently published in peer-reviewed journals.⁴ Furthermore, even in abstracts describing case reports, a past study reported that more than 30% of them are subsequently published.⁵ Therefore, this explanation does not fully account for our results. Third, given that a lower quality of abstracts was significantly associated with a lower publication rate,²⁴ abstracts that should be rejected due to low quality might be accepted without rejection in JPCAM. However, considering the extent of effects of abstract quality on the publication rate,⁴ a lower quality of abstracts is also unlikely to account for the extremely low publication rate. Fourth, a past systematic review reported that the most frequently cited barriers to abstract publication were a lack of time,^25–29 although all studies included in that systematic review were conducted outside Japan.²⁹ However, in the post hoc analysis of this study, even among abstracts of first authors who presented two or more abstracts as a first author at JPCAMs during 2010–2012, only 4.3% of their abstracts were subsequently published in a peer-reviewed journal. This means that these authors did not take the time to publish their abstracts, although these authors had enough time to prepare for submitting two or more abstracts to the conference. Therefore, a lack of time also seems unlikely to explain our findings.

Finally, given that known predictive factors associated with publications among the characteristics of the presented abstracts and affiliations of the authors could not account for our findings, other unknown predictive factors are considered to highly affect the abstract publication. A systematic review reported that some authors had no reason for not publishing abstracts in a peer-reviewed journal or stated that publication was not an aim.²⁹ Therefore, personal factors might affect abstract publication. The presentation of abstracts at a conference might itself have been an end goal for Japanese primary care practitioners. Given that this personal factor might strongly affect abstract publication, a prospective study evaluating the effect of personal factors of authors on abstract publication is warranted. For example, evaluating the purpose of presentation or the attitudes about abstract publication when abstracts are submitted to the conference and investigating the subsequent publication of their abstracts may be useful. Nonetheless, other unknown predictive factors associated with abstract publication also need to be investigated in the future, as this study did not evaluate the personal factors of authors.

To disseminate important scientific information beyond the conference, some strategies to increase the publication rate of abstracts presented at the conference are needed. Among five variables (year of the annual meeting, presentation format, affiliation of the first author, number of authors involved and number of authors’ affiliations) examined in this study, oral presentation and first author affiliation with a university-associated institution were the only independent predictive factors for publication. Our findings are consistent with those of past studies.4 6 15 22 30–32 Therefore, one strategy to increase abstract publication is to increase the proportion of oral presentations and strengthen the partnership of universities with primary care practices, although it remains unclear why abstracts of oral presentation are more likely to be published than abstracts of poster presentations.⁶ However, even in abstracts presented as oral presentations by the first author from a university-associated institution, no more than 12% of abstracts were published in peer-reviewed journals in this study. Therefore, this strategy seems insufficient for a conference where the publication rate of abstracts is extremely low, such as JPCAM. Given that personal factors of authors may strongly affect abstract publication, as we noted above, some incentives or obligations for authors regarding abstract publication may be needed. Another strategy is to divide submitted abstracts into two sections of the conference according to whether or not authors plan to publish them in peer-reviewed journals. Although the purpose of this method is not to increase the abstract publication, this distinction is important for the audience of the conference because abstracts that are not received in a peer-reviewed process can mislead the audience due to their poor quality or descriptions.^{33 34} To improve the publication rate of abstracts, further studies investigating the barriers to publication among investigators who participate in conferences where the publication rate is extremely low are needed.

Strengths and weaknesses of the study

To our knowledge, this study is the first to determine the publication rate of abstracts presented at JPCAMs. Furthermore, while most past studies did not evaluate or report interobserver reliability for searches for publications corresponding to presented abstracts,1–5 15 18 19 21–23 25 30–32 35–37 we evaluated the interobserver reliability for the outcome assessment.

Several limitations of this study must be mentioned. First, publication status was determined based on a single database. Furthermore, we did not contact any authors of the abstracts to obtain information about publication. Therefore, the publication rate of abstracts might be underestimated. However, based on past studies,^{26 33} contacting the authors of abstracts increases the observed abstract publication rate by no more than 12%. Even if such an increase were considered, the publication rate of abstracts presented at JPCAMs is still much lower than that of annual scientific meetings for primary care in other countries, in which the publication rate of abstracts is more than 40%.1 4 5 15 Second, search error due to misspelling of the author’s name might result in underestimation of the publication rate.³⁸ Third, factors that might be associated with publication, such as abstract results,⁴ abstract quality²⁴ and research design,⁴ were not evaluated. Fourth, rejected abstracts were not included because information on rejected abstracts was not disclosed by JPCAM. Fifth, the limited follow-up period might restrict the time frame of searchable articles. Some abstracts may be published more than 5 years after presentation at the corresponding annual meeting.⁴ Furthermore, publication before the conference was excluded, although there were only three abstracts identified to have been published before the conference in this study. Sixth, personal factors of authors, such as their attitudes about abstract publication, were not evaluated. Finally, a past study reported that there were often various inconsistencies between abstracts presented at annual meetings and corresponding published articles,³³ and for some abstracts included in this study, judging whether they were the same work as an identified article was difficult.

Conclusions

The publication rate of abstracts presented at JPCAMs was extremely low. Given that known predictive factors associated with publication among the characteristics of presented abstracts and authors’ affiliation could not account for our findings, other unknown predictive factors might strongly affect the abstract publication rate. Further studies are warranted to investigate the barriers to publication among investigators who participate in conferences with an associated publication rate that is extremely low.