Be careful what you believe

The authors of this study should be commended for taking a critical look at the quality of their local urology meeting as represented by the ultimate publication rates of the abstracts presented.¹ The ultimate publication rate of 56% from the QUA meeting is congruent with publication rates from other major urology meetings, including the American Urological Association meeting, which has been demonstrated to have a publication rate of 55%.²

It is interesting that non-Quebec institutions had a higher publication rate than local institutions. Perhaps adding context to this was that the authors noted that 70% of the presentations were of research conducted by residents or fellows. I suspect there is a selection bias at play here, as it is well-known that most training programs have an expectation of trainees to maintain a minimum level of academic productivity (creating, albeit inadvertently, an emphasis on quantity over quality, which may translate to a lower ultimate publication rate). I suspect that authors coming from further afield to present their research do so after a level of self-selection (they are proud of their research and feel the findings should be shared, this motivation possibly later translating into higher publication rates).

Publications rates from multiple medical meetings have been investigated by many authors. As again reflected in this paper, we know that a significant proportion of research presented at scientific meetings either is never subjected to or ultimately fails the peer-review process and is thus never published. We know that meeting abstracts themselves are frequently cited in other published papers,³ thereby lending these potentially unpublished abstracts a legitimacy they may not deserve.

The critical question at hand is whether presentations at scientific meetings have any impact on patient care. Several authors have investigated the impact of presentations at scientific meetings on subsequent clinical practice in advance of peer-reviewed publication of that research with frightening findings. Gross and colleagues demonstrated that carotid endarterectomy rates jumped far more significantly in the first few months after prepublication release of the results of 2 seminal trials on this subject than after final publication of these trials.⁴ Even more convincing for the potential of unpublished scientific presentations affecting clinical practice was the dramatic increase in paclitaxel chemotherapy use for metastatic breast cancer after presentation of the Cancer and Leukemia Group B (CALGB) Study 9344 at the American Society of Clinical Oncology meeting May 1998. In the months preceding this meeting, 5.2% of women in this patient population received placlitaxel as opposed to 23.6% in the months immediately following the meeting (before study publication and FDA approval).⁵ While these 2 clinical interventions have stood the test of time, these papers demonstrate the potential impact of unpublished (not yet peer-reviewed) research on clinical practice. This is sobering when one considers that about 50% of presented abstracts at major medical meetings never make it through the peer-review process.

Which leads me to my concluding advice: the next time you go to a medical meeting, bring a coin — you’re going to have to flip it to decide whether to believe what you hear...or not.