Abstract
Background
Abstracts accepted at scientific meetings are often not subsequently published. Data on publication rates are largely from subspecialty and surgical studies.
Objective
The aims of this study were to 1) determine publication rates of abstracts presented at a general internal medicine meeting; 2) describe research activity among academic general internists; 3) identify factors associated with publication and with the impact factor of the journal of publication; and 4) evaluate for publication bias.
Design
Retrospective cohort study.
Participants
All scientific abstracts presented at the Society of General Internal Medicine 2009 Annual Meeting.
Main Measures
Publication rates were determined by searching for full-text publications in MEDLINE. Data were abstracted regarding authors’ institution, research topic category, number of study sites, sample size, study design, statistical significance (p value and confidence interval) in abstract and publication, journal of publication, publication date, and journal impact factor.
Key Results
Of the 578 abstracts analyzed, 274 (47.4%) were subsequently published as a full article in a peer-reviewed journal indexed in MEDLINE. In a multivariable model adjusting for institution site, research topic, number of study sites, study design, sample size, and abstract results, publication rates for academic general internists were highest in the areas of medical education (52.5%, OR 5.05, 95% CI 1.57–17.25, reference group Veterans Affairs (VA)-based research, publication rate 36.7%), mental health/substance use (67.7%, OR 4.16, 95% CI 1.39–13.06), and aging/geriatrics/end of life (65.7%, OR 3.31, 95% CI 1.15–9.94, p = 0.01 across topics). Publication rates were higher for multicenter studies than single-institution studies (52.4% vs. 40.4%, OR 1.66, 95% CI 1.10–2.52, p = 0.04 across categories). Randomized controlled trials had higher publication rates than other study designs (66.7% vs. 45.9%, OR 2.72, 95% CI 1.30–5.94, p = 0.03 across study designs). Studies with positive results did not predict higher publication rates than negative studies (OR 0.89, 95% CI 0.6–1.31, p = 0.21).
Conclusions
This study demonstrated that 47.4% of abstracts presented at a general internal medicine national conference were subsequently published in a peer-reviewed journal indexed in MEDLINE.
KEY WORDS: general internal medicine, abstracts, publication rates, impact factor, publication bias
INTRODUCTION
While research presentations at scientific meetings are often the initial method of sharing new research findings among the academic community, peer-reviewed publications are essential for the validation and dissemination of research findings. Despite this, studies suggest that more than half of the abstracts accepted to national meetings are never published; a Cochrane review demonstrated a mean publication rate of 44.5% across medical and surgical specialties.1 Of the 79 reports identified by the Cochrane review, only two studies were classified as focusing on internal medicine.1 Upon further review of these two studies, the vast majority of abstracts analyzed were found to be from subspecialty meetings. The first study, published in the Australian and New Zealand Journal of Medicine, included abstracts from 17 different society meetings, the majority of which were subspecialty meetings.2 The second study, published in Nederlands Tijdschrift voor Geneeskunde, included abstracts from 54 meetings comprising surgery, medicine, and non-clinical medicine.3 Outside of this review, we identified one published study from general internist research, but this study focused on research in medical education.4 None of the identified reports have exclusively identified publication rates from all abstracts presented at a meeting of general internists.
Academic general internal medicine is a relatively new field that has grown substantially over the last several decades.5 General internist researchers tend to investigate broad medical areas that do not fit neatly into other medical subspecialties, or topics that span many subspecialties, such as patient–physician communication, end-of-life care, medical ethics, health care delivery, health economics, and chronic disease management.6 Academic internists have participated in the foundation of research in evidence-based medicine, patient–physician communication, and health outcomes.7 The Society of General Internal Medicine (SGIM) is a national medical society, and is the primary academic home for academic general internists, with over 3000 active members. An SGIM task force report defined the role of general internists, recommending that internists engage in research focused on “high-value, comprehensive, and longitudinal care” in the areas of chronic disease management, prevention, health care quality, cost control/effectiveness, population health, doctor–patient communication, and medical education.7
Given the importance of academic general internists in advancing important areas of research, and considering that the rate of abstract conversion to publication for general internists is unknown, we sought to determine the peer-reviewed publication rates of scientific abstracts initially presented at an SGIM annual meeting and to describe the research topics that academic general internists emphasize and publish. The SGIM annual meeting is the premier meeting for academic general internists to present a wide spectrum of clinical and translational medicine and medical education research.8 We also sought to determine factors associated with publication in general and with higher-impact publications in the setting of general internal medicine research. Lastly, given the importance of publishing negative results, especially in the setting of aggregation of data into systematic reviews and meta-analyses, we examined whether there was publication bias favoring abstracts reporting statistically significant results.
METHODS
We conducted a retrospective cohort study of abstracts submitted to the SGIM 2009 annual meeting. We chose the 2009 meeting based on a review of the literature, as research has shown that more than 95% of published articles are published within 5 years of presentation,1 and most publications occur within 3 years of presentation.9 , 10 For our analysis, we included all abstracts of original research and excluded all case vignettes and innovations in medical education.
For the primary outcome, we determined subsequent publication by searching keywords from the title and author in the MEDLINE database. If this approach did not yield results, we searched for various combinations of keywords in the title alone or in combination with the authors’ names. The title, methods, and results of the published article were compared to the original abstract to confirm the corresponding publication. Research was considered published if we identified a matching full-length article using this search strategy. We considered a “brief report” a published article, because it is subject to peer review and indexed in MEDLINE. Research was considered unpublished if we were unable to obtain results via either search method. We completed the search in March 2015.
To describe the research breadth of academic general internists, we also extracted the submission category that was chosen by the author for presentation at the national SGIM meeting. We analyzed the presentation frequency and publication rates for these topics. We examined other potential associations with subsequent publication, including institution (community, academic), number of study sites (single, multicenter U.S., multicenter international), sample size, study design (randomized controlled trials, systematic reviews/meta-analyses, and "other"—defined as non-experimental studies, including observational, case–control, and qualitative studies), statistical significance (p value and confidence interval) in abstract and final publication, journal of publication, and publication date. We determined the impact factor of the journal of the published article using Clarivate Analytics Web of Science, extracting the impact factor in the year the article was published.
To evaluate for publication bias, we defined positive results as statistically significant results, with any reported p value less than 0.05 for any of the tested hypotheses. We examined all abstracts and published articles for the presence or absence of statistically significant results.
Multiple logistic regression was applied to evaluate a multivariable model of factors associated with publication. All tested covariates—institution site, research topic, number of study sites, study design, sample size and abstract results—were included in this model. The association between these covariates and journal impact factor for published manuscripts was assessed by nonparametric analysis of variance (Kruskal–Wallis) tests, given the skewed nature of journal impact factors. Statistical significance was set at p < 0.05 in all analyses. Statistical analyses were conducted using SAS Version 9.3 software (SAS Institute Inc., Cary, NC). This study was deemed exempt by the Mayo Clinic Institutional Review Board.
RESULTS
Of the 578 analyzed abstracts, 274 (47.4%) were subsequently published as a full article in a peer-reviewed journal within 5 years of presentation, and 235 (40.7%) abstracts were published within 3 years of presentation. The mean time to publication was 19.5 months.
The most common submission categories were health disparities/vulnerable populations (15.2%), medical education (13.8%), organization of care and chronic disease management (12.1%), clinical epidemiology/health care effectiveness research (10.9%), and quality of care/patient safety (9.5%). Together, these five categories represent over 60% of the abstracts presented. However, the topical areas which were most likely to be published included medical education (publication rate 52.5%, OR 5.05, 95% CI 1.57–17.25, reference group Veterans Affairs (VA)-based research, publication rate 36.7%), mental health/substance use (67.7%, OR 4.16, 95% CI 1.39–13.06), and aging/geriatrics/end of life (65.7%, OR 3.31, 95% CI 1.15–9.94, p = 0.01 across topics).
In multivariable analyses, only the number of study sites and the study design were associated with higher publication rates (Table 1). Publication rates were higher for U.S. multicenter studies than single institutions (52.4% vs. 40.4%, OR 1.66, 95% CI 1.10–2.52, p = 0.04). Randomized controlled trials were more likely to be published than other study designs (66.7% vs. 45.9%, OR 2.72, 95% CI 1.30–5.94, p = 0.03), while systematic reviews/meta-analyses were not more likely to be published than other study designs (50.0% vs. 45.9%, OR 1.23, 95% CI 0.33–4.44). Publication rates were not associated with sample size or type of institution (academic vs. community).
Table 1.
Multivariable Results from Logistic Regression for Outcome of Publication of Abstracts Submitted to the 2009 SGIM Annual Meeting
| Variable | N (%) | % Published | Odds ratio | 95% Confidence interval | p | |
|---|---|---|---|---|---|---|
| Site | Community | 24 (4.2%) | 25.0% | Ref. | Ref. | 0.051 |
| Academic | 554 (95.8%) | 48.4% | 2.58 | 0.997–7.60 | ||
| Topic | VA-based research | 30 (5.2%) | 36.7% | Ref. | Ref. | 0.01 |
| Health disparities/vulnerable populations | 88 (15.2%) | 42.0% | 1.34 | 0.54–3.37 | ||
| Medical education | 80 (13.8%) | 52.5% | 5.05 | 1.57–17.25 | ||
| Organization of care and chronic disease management | 70 (12.1%) | 44.3% | 1.28 | 0.50–3.31 | ||
| Clinical epidemiology/health care effectiveness research | 63 (10.9%) | 44.4% | 1.38 | 0.53–3.61 | ||
| Quality of care/patient safety | 55 (9.5%) | 45.5% | 1.45 | 0.55–3.85 | ||
| Aging/geriatrics/end of life | 35 (6.1%) | 65.7% | 3.31 | 1.15–9.94 | ||
| Preventive medicine | 32 (5.5%) | 37.5% | 1.01 | 0.33–3.03 | ||
| Mental health/substance abuse | 31 (5.4%) | 67.7% | 4.16 | 1.39–13.06 | ||
| Clinical decision-making and economic analysis | 19 (3.3%) | 57.9% | 2.50 | 0.72–8.97 | ||
| Women’s health | 15 (2.6%) | 66.7% | 3.22 | 0.85–13.42 | ||
| Cancer research | 12 (2.1%) | 41.7% | 1.11 | 0.25–4.54 | ||
| Global health/preparedness | 10 (1.7%) | 40.0% | 0.62 | 0.07–4.70 | ||
| Other* | 38 (6.6%) | 36.8% | 1.06 | 0.37–3.02 | ||
| Number of study sites | Single site | 235 (40.7%) | 40.4% | Ref. | Ref. | 0.04 |
| Multicenter, USA | 330 (57.1%) | 52.4% | 1.66 | 1.10–2.52 | ||
| Multicenter, international | 13 (2.2%) | 46.2% | 2.73 | 0.50–16.85 | ||
| Study design | Other† | 527 (91.2%) | 45.9% | Ref. | Ref. | 0.03 |
| Randomized controlled trial | 39 (6.7%) | 66.7% | 2.72 | 1.30–5.94 | ||
| Systematic review/meta-analysis | 12 (2.1%) | 50.0% | 1.23 | 0.33–4.44 | ||
| Sample size | ≤50 | 74 (12.8%) | 41.9% | Ref. | Ref. | 0.07 |
| 51–100 | 58 (10.0%) | 50.0% | 1.27 | 0.61–2.67 | ||
| 101–250 | 105 (18.2%) | 41.0% | 0.87 | 0.44–1.69 | ||
| 251–500 | 93 (16.1%) | 48.4% | 1.28 | 0.63–2.58 | ||
| 501–2000 | 84 (14.5%) | 51.2% | 1.36 | 0.66–2.82 | ||
| >2000 | 128 (22.1%) | 57.8% | 1.74 | 0.87–3.48 | ||
| Unknown | 36 (6.2%) | 25.0% | 0.50 | 0.18–1.25 | ||
| Abstract result | Negative | 247 (42.7%) | 46.6% | Ref. | Ref. | 0.21 |
| Positive | 276 (47.8%) | 48.6% | 0.89 | 0.60–1.31 | ||
| Not Applicable | 55 (9.5%) | 45.5% | 0.39 | 0.13–1.11 |
*“Other” topics include health policy/advocacy/social justice, hospital-based medicine, medical humanities and ethics, personal/professional development, and research methods
†“Other” study designs were defined as non-experimental studies, including observational, case–control, and qualitative studies
We assessed the impact of publications according to institution specified in the abstract, topic, number of study sites, study design, sample size, and abstract result (Table 2). There were significant differences in the impact factor of the journal of publication based on the submission category, with the highest median impact factors identified for cancer research (19.0), quality of care and patient safety (3.9), aging/geriatrics/end of life (3.9), and preventative medicine (3.9). The median impact factor was higher for multicenter trials than for single-site studies (3.2 vs. 2.6, p = 0.002). Systematic reviews/meta-analyses were associated with a higher median impact factor than randomized controlled trials and observational studies (8.2 vs. 3.9 vs. 2.8, p = 0.008). There was no association between median impact factor and type of institution (academic vs. community) or abstract result (positive vs. negative).
Table 2.
Impact Factors* for Publications Resulting from Abstracts Submitted to the 2009 SGIM Annual Meeting
| Variable | n | Median impact factor | p § | |
|---|---|---|---|---|
| Site | Academic | 268 | 2.8 | 0.63 |
| Community | 6 | 2.5 | ||
| Topic | VA-based research | 11 | 3.0 | 0.007 |
| Health disparities/vulnerable populations | 37 | 3.2 | ||
| Medical education | 42 | 2.6 | ||
| Organization of care and chronic disease management | 31 | 2.8 | ||
| Clinical epidemiology/health care effectiveness research | 28 | 2.8 | ||
| Quality of care/patient safety | 25 | 3.9 | ||
| Aging/geriatrics/end of life | 23 | 3.9 | ||
| Preventive medicine | 12 | 3.9 | ||
| Mental health/substance abuse | 21 | 3.0 | ||
| Clinical decision-making and economic analysis | 11 | 3.1 | ||
| Women’s health | 10 | 2.8 | ||
| Cancer research | 5 | 19.0 | ||
| Global health/preparedness | 4 | 1.3 | ||
| Other† | 14 | 3.9 | ||
| Number of study sites | Single site | 95 | 2.6 | 0.002 |
| Multicenter, USA | 173 | 3.2 | ||
| Multicenter, international | 6 | 1.5 | ||
| Study design | Other‡ | 242 | 2.8 | 0.008 |
| Randomized controlled trial | 26 | 3.9 | ||
| Systematic review/meta-analysis | 5 | 8.2 | ||
| Sample size | ≤50 | 31 | 2.3 | 0.002 |
| 51–100 | 29 | 2.1 | ||
| 101–250 | 43 | 3.4 | ||
| 251–500 | 45 | 3.1 | ||
| 501–2000 | 43 | 3.3 | ||
| >2000 | 74 | 3.2 | ||
| Unknown | 9 | 2.4 | ||
| Abstract result | Negative | 115 | 2.8 | 0.05 |
| Positive | 134 | 3.3 | ||
| Not applicable | 25 | 2.6 |
*The impact factor was extracted from the year the article was published
†“Other” topics include health policy/advocacy/social justice, hospital-based medicine, medical humanities and ethics, personal/professional development, and research methods
‡“Other” study designs were defined as non-experimental studies, including observational, case–control, and qualitative studies
§Nonparametric analysis of variance (Kruskal–Wallis) test
Abstracts with positive and negative results had similar publication rates (OR 0.89, 95% CI 0.60–1.31, p = 0.21; Table 1). Of the 134 abstracts that were subsequently published with positive results, the subsequently published article also reported positive results in 120 cases (89.6%). Of the 115 abstracts reporting negative results, the subsequently published article also reported negative results in 91 cases (79.1%). The median impact factor of positive-result abstracts versus negative results was not statistically different (3.3 vs. 2.8, p = 0.05; Table 2).
DISCUSSION
We identified an overall publication rate of 47.4% within 5 years of presentation; given the relative youth of the field of academic general internal medicine, this finding is reassuring and is consistent with previous research on publication rates in other fields.1 Regarding the content areas in which academic general internists present and publish, we found that health disparities, medical education, and chronic disease management were the most frequently presented topics, and that research in medical education, mental health, and geriatrics had the highest publication rates. Multicenter studies were more likely to be published than single-site studies, and randomized controlled trials had higher rates of publication than systematic reviews/meta-analyses, and observational studies. Additionally, we identified associations between the impact factor of publication and number of study sites, study design, sample size, topic of research, and abstract result. We did not identify publication bias, as abstracts with positive and negative results were equally likely to be published.
Our findings support the diversity of publications and important research contributions by general internists across biomedical research, including health disparities, medical education, chronic disease management, health care effectiveness, and quality of care. These topics correspond to emerging research needs in the United States, as identified in the statement by the SGIM task force.7 While general internists are presenting research on these important topics, however, most of these abstract categories reached publication less than 50% of the time. This would imply that although general internists have identified and addressed important gaps in research, barriers to sharing this research via publication remain. Based on survey data, major reasons for failure to publish include authors’ failure to submit,11 – 14 with lack of time, funding, or other resources being the most frequently cited contributing factors.13 , 15 , 16
Our findings support prior research showing that multi-institutional collaborations are more likely to publish than single-institution studies.12 Our results were also similar to those of previous studies in that randomized controlled trials were more likely to be published than other study designs,1 and meta-analyses were associated with the highest citation index.17 These findings may reflect the emphasis among the scientific community regarding the hierarchy of evidence, with rigorous meta-analyses receiving the highest status. Sample size was not predictive of publication in our study. Previous studies have shown mixed results as to the influence of sample size on publication.1 , 9 , 12 , 15 Earlier studies have dichotomized the sample size to above or below the median sample size of the included abstracts, and have shown minimal impact of sample size on publication.1 Because of the range of sample sizes in our study, we were able examine this relationship more closely, and we found no influence of sample size on publication across a broad range of sample sizes.
Authors from academic institutions did not publish more often than those from community centers. Academic institutions have traditionally offered more protected time for research activities than have community practices, and this finding may reveal a trend towards the increasing clinical demands of academic practice.18 Alternatively, many community centers have institutional review boards that may facilitate research among community physicians at levels similar to what can be achieved by general internal medicine physicians at academic institutions.
Publication bias is a well-known phenomenon. Studies with statistically significant findings are much more likely to be published.1 , 10 – 12 , 15 , 16 , 19 – 22 Unfortunately, exclusion of negative studies from the literature hides important scientific information from readers, creates imbalance in the overall body of evidence, and impairs the robustness of systematic reviews and meta-analyses. Reassuringly, we did not identify publication bias among general internal medicine research. This may reflect a greater understanding of publication bias and calls for increasing clinical trial registration23 and placing importance on publishing negative studies.24 Additionally, this may reflect the generalists’ perspective, which, for example, stresses comparative effectiveness research over comparisons of active interventions with placebo.
A recent paper in the American Journal of Surgery examining publication rates from the Dutch Surgical Society proposed using a conference’s abstract-to-publication ratio as a quality indicator for scientific meetings to help in assessing scientific value and educational quality.25 The current study showcases the variety of clinical, translational, and medical education research presented at the SGIM annual meeting and demonstrates comparable publication rates. While quality of a scientific meeting is multifaceted, studies like this can identify a scientific meeting’s publication conversion rates and factors associated with publication, and may help medical societies identify factors that lead to greater success in publishing among investigators, while bolstering the scientific impact of their annual meetings.
Our study has limitations. Prior research has verified authors’ publications through various means of personal contact, thus revealing additional published studies not indexed in MEDLINE. This study represents a restricted time frame of the searchable literature; some studies may have been published more than 5 years after manuscript submission. Nonetheless, previous research has shown that the vast majority of studies will be published within the first 3 years after submission.9 , 10 We included abstracts from just one meeting, which precludes the ability to follow time trends. Publication bias may include additional aspects that were not addressed by this study, including primary study aim, author’s notation of “positive” association, and/or statistically significant results.
CONCLUSION
We found that publication rates and time to publication for abstracts accepted to a general internal medicine national meeting were similar to what has been reported for surgical and subspecialty meetings.1 This study indicates that general internists contribute a broad variety of topics to the biomedical literature. Journal publication rates and impact factors were associated with the number of study sites, study design, and sample size. These factors are important to consider when designing studies for publication. We did not identify publication bias in the abstracts that were subsequently published. An important direction for future studies will be to determine factors associated with failure to publish, as this would provide insight into why more than 50% of abstracts remain unpublished.
Acknowledgements
This study was originally presented as an oral abstract presentation at the Society of General Internal Medicine 2016 Annual Meeting on May 13, 2016.
Compliance with Ethical Standards
Conflict of Interest
The authors declare no conflicts of interest.
References
- 1.Scherer RW, Langenberg P, von Elm E. Full publication of results initially presented in abstracts. Cochrane Database Syst Rev. 2007;(2):MR000005. [DOI] [PubMed]
- 2.Hopewell S, McDonald S. Full publication of trials initially reported as abstracts in the Australian and New Zealand Journal of Medicine 1980–2000. Intern Med J. 2003;33(4):192–4. doi: 10.1046/j.1445-5994.2003.00353.x. [DOI] [PubMed] [Google Scholar]
- 3.Koene HR, Overbeke AJ. The scientific value of society reports in the Nederlands Tijdschrift voor Geneeskunde. Ned Tijdschr Geneeskd. 1994;138(37):1868–71. [PubMed] [Google Scholar]
- 4.Sawatsky AP, Beckman TJ, Edakkanambeth Varayil J, Mandrekar JN, Reed DA, Wang AT. Association between study quality and publication rates of medical education abstracts presented at the society of general internal medicine annual meeting. J Gen Intern Med. 2015;30(8):1172–7. doi: 10.1007/s11606-015-3269-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Armstrong K, Keating NL, Landry M, Crotty BH, Phillips RS, Selker HP. Academic general internal medicine: a mission for the future. J Gen Intern Med. 2013;28(6):845–51. doi: 10.1007/s11606-013-2334-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Levinson W, Linzer M. What is an academic general internist? Career options and training pathways. JAMA. 2002;288(16):2045–8. doi: 10.1001/jama.288.16.2045. [DOI] [PubMed] [Google Scholar]
- 7.Larson EB, Fihn SD, Kirk LM, Levinson W, Loge RV, Reynolds E, Sandy L, Schroeder S, Wenger N, Williams M. The future of general internal medicine. Report and recommendations from the Society of General Internal Medicine (SGIM) Task Force on the Domain of General Internal Medicine. J Gen Intern Med. 2004;19(1):69–77. doi: 10.1111/j.1525-1497.2004.31337.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Society of General Internal Medicine. Available at http://www.sgim.org/about-us. Accessed 15 Aug 2016.
- 9.Scherer RW, Dickersin K, Langenberg P. Full publication of results initially presented in abstracts. A meta-analysis. JAMA. 1994;272(2):158–62. doi: 10.1001/jama.1994.03520020084025. [DOI] [PubMed] [Google Scholar]
- 10.Eloubeidi MA, Wade SB, Provenzale D. Factors associated with acceptance and full publication of GI endoscopic research originally published in abstract form. Gastrointest Endosc. 2001;53(3):275–82. doi: 10.1016/S0016-5107(01)70398-7. [DOI] [PubMed] [Google Scholar]
- 11.Easterbrook PJ, Berlin JA, Gopalan R, Matthews DR. Publication bias in clinical research. Lancet. 1991;337(8746):867–72. doi: 10.1016/0140-6736(91)90201-Y. [DOI] [PubMed] [Google Scholar]
- 12.Dickersin K, Min YI, Meinert CL. Factors influencing publication of research results. Follow-up of applications submitted to two institutional review boards. JAMA. 1992;267(3):374–8. doi: 10.1001/jama.1992.03480030052036. [DOI] [PubMed] [Google Scholar]
- 13.Weber EJ, Callaham ML, Wears RL, Barton C, Young G. Unpublished research from a medical specialty meeting: why investigators fail to publish. JAMA. 1998;280(3):257–9. doi: 10.1001/jama.280.3.257. [DOI] [PubMed] [Google Scholar]
- 14.Sprague S, Bhandari M, Devereaux PJ, Swiontkowski MF, Tornetta P, 3rd, Cook DJ, Dirschl D, Schemitsch EH, Guyatt GH. Barriers to full-text publication following presentation of abstracts at annual orthopaedic meetings. J Bone Joint Surg Am. 2003;85-A(1):158–63. doi: 10.2106/00004623-200301000-00024. [DOI] [PubMed] [Google Scholar]
- 15.De Bellefeuille C, Morrison CA, Tannock IF. The fate of abstracts submitted to a cancer meeting: factors which influence presentation and subsequent publication. Ann Oncol. 1992;3(3):187–91. doi: 10.1093/oxfordjournals.annonc.a058147. [DOI] [PubMed] [Google Scholar]
- 16.Krzyzanowska MK, Pintilie M, Tannock IF. Factors associated with failure to publish large randomized trials presented at an oncology meeting. JAMA. 2003;290(4):495–501. doi: 10.1001/jama.290.4.495. [DOI] [PubMed] [Google Scholar]
- 17.Patsopoulos NA, Analatos AA, Ioannidis JP. Relative citation impact of various study designs in the health sciences. JAMA. 2005;293(19):2362–6. doi: 10.1001/jama.293.19.2362. [DOI] [PubMed] [Google Scholar]
- 18.Pollart SM, Novielli KD, Brubaker L, Fox S, Dandar V, Radosevich DM, Misfeldt ML. Time well spent: the association between time and effort allocation and intent to leave among clinical faculty. Acad Med. 2015;90(3):365–71. doi: 10.1097/ACM.0000000000000458. [DOI] [PubMed] [Google Scholar]
- 19.Weintraub WH. Are published manuscripts representative of the surgical meeting abstracts? An objective appraisal. J Pediatr Surg. 1987;22(1):11–3. doi: 10.1016/S0022-3468(87)80005-2. [DOI] [PubMed] [Google Scholar]
- 20.Dickersin K. The existence of publication bias and risk factors for its occurrence. JAMA. 1990;263(10):1385–9. doi: 10.1001/jama.1990.03440100097014. [DOI] [PubMed] [Google Scholar]
- 21.Callaham ML, Wears RL, Weber EJ, Barton C, Young G. Positive-outcome bias and other limitations in the outcome of research abstracts submitted to a scientific meeting. JAMA. 1998;280(3):254–7. doi: 10.1001/jama.280.3.254. [DOI] [PubMed] [Google Scholar]
- 22.von Elm E, Costanza MC, Walder B, Tramèr MR. More insight into the fate of biomedical meeting abstracts: a systematic review. BMC Med Res Methodol. 2003;3:12. doi: 10.1186/1471-2288-3-12. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.De Angelis C, Drazen JM, Frizelle FA, Haug C, Hoey J, Horton R, Kotzin S, Laine C, Marusic A, Overbeke AJ, Schroeder TV, Sox HC, Van Der Weyden MB. Clinical trial registration: a statement from the International Committee of Medical Journal Editors. Ann Intern Med. 2004;141(6):477–8. doi: 10.7326/0003-4819-141-6-200409210-00109. [DOI] [PubMed] [Google Scholar]
- 24.Sridharan L, Greenland P. Editorial policies and publication bias: the importance of negative studies. Arch Intern Med. 2009;169(11):1022–3. doi: 10.1001/archinternmed.2009.100. [DOI] [PubMed] [Google Scholar]
- 25.de Meijer VE, Knops SP, van Dongen JA, Eyck BM, Vles WJ. The fate of research abstracts submitted to a national surgical conference: a cross-sectional study to assess scientific impact. Am J Surg. 2016;211(1):166–71. doi: 10.1016/j.amjsurg.2015.06.017. [DOI] [PubMed] [Google Scholar]