Pharmacy journal abstracts published in PubMed that abide by the CONsolidated Standards Of Reporting Trials (CONSORT) guidelines

Abstract

The purpose of this research was to determine the proportion of abstracts in pharmacy journals that are prepared according to the CONsolidated Standards Of Reporting Trials (CONSORT) criteria for abstracts. Certain abstracts for randomized controlled clinical trials (RCTs) indexed in PubMed were eligible for inclusion, with the primary endpoint being median overall compliance to CONSORT recommendations for abstracts. A total of 63 RCT abstracts were included in the analysis, with only 56% of the recommended CONSORT items represented in the sample. It is recommended that pharmacy journals encourage authors to follow CONSORT recommendations for abstracts when submitting RCTs for publication.

INTRODUCTION

The volume of medical and pharmacy literature published annually is staggering, and the ability of health care providers to efficiently search and remain current with health literature is becoming increasingly difficult. One source that many practitioners utilize is PubMed, developed by the National Center for Biotechnology Information (NCBI) at the National Library of Medicine (NLM). PubMed currently indexes more than 19 million references from more than 5,600 journals. This resource alone adds between 2,000 to 4,000 references weekly to its repository 1.

One method to cope with the vast amount of published information that many practitioners use to stay current with new clinical research and findings is to review the abstract of a journal article before fully committing to reading the full-text article. Although not advocated as good practice, results of past studies indicate that health care practitioners rely heavily on abstracts 2, 3. For instance, a 2001 survey of family physicians concluded that doctors were willing to alter their current practice, at a rate as high as 70%, based solely on data presented in an abstract 3.

Using only the abstract to make decisions is worrisome. Abstracts are not designed to provide the reader with all of the study details. But more importantly, issues can be present with abstracts. For instance, Ward et al. conducted a study reporting that up to 33% of abstracts published in major pharmacy journals contained inaccuracies or omissions 4. Up to 60% of the reviewed abstracts were classified as deficient 4. In addition, discrepancies between journal abstracts and full-text articles in major general medical journals (e.g., New England Journal of Medicine, BMJ) have been reported, with the proportion of discrepancies in abstracts in different journals ranging from 18% to 68% 5. Furthermore, investigators examining abstracts in 8 psychology journals reported deficiencies that ranged from 8% to 18%, in which major errors (defined as errors that could affect the interpretation of the article) occurred in 37% of deficient abstracts 6.

Due to the potential for patient harm from errant medical decisions supported by data from inaccurate abstracts, emphasis has been placed on abstracts of randomized controlled trials (RCTs) being accurate and sufficiently detailed to allow for quick review of the information that is contained in the full-text article 7, 8. In an effort to improve the detail and clarity of abstracts, the CONsolidated Standards Of Reporting Trials (CONSORT) group published an extension to their 2001 guidelines 9 for reporting RCTs that specifically addressed the reporting of journal and conference abstracts. This extension, published in January 2008, provides a sixteen-point checklist (seventeen-point checklist for conference abstracts) of items that should be considered for inclusion when reporting an RCT in a journal abstract 7. The CONSORT recommendations have been endorsed by the World Association of Medical Editors, the International Committee of Medical Journal Editors, and the Council of Science Editors 7.

The CONSORT recommendations were published in order to aid authors in preparing high-quality abstracts, but studies examining the adherence to the recommendations have indicated suboptimal reporting of trial data in a wide variety of medical fields 10–16. However, no studies have reported on the compliance of pharmacy journals to the CONSORT recommendations. Therefore, the purpose of this study was to compare the content of abstracts in journal articles to the guidelines to determine how many published abstracts follow these recommendations.

METHODS

Study selection

All RCT abstracts indexed in PubMed and published between January 1, 2009, and December 31, 2011, in five major pharmacy journals were included. The specific journal titles utilized were: Pharmacotherapy: The Journal of Human Pharmacology and Drug Therapy (Pharmacotherapy), Journal of the American Pharmacists Association (J Am Pharm Assoc), Annals of Pharmacotherapy (Ann Pharmacother), American Journal of Health-System Pharmacy (Am J Health Syst Pharm), and Consultant Pharmacist (Consult Pharm). These journal titles represent the official publications of major pharmacy associations and/or routinely publish RCTs, and are indexed in PubMed. The publication date range of January 1, 2009, to December 31, 2011, allowed comparison between three years of abstracts and ensured sufficient time for authors and editors to incorporate the recommendations into practice following the publication of the CONSORT abstract recommendations in January 2008.

All trials that met the inclusion criteria (indexed in PubMed as an RCT; published between January 1, 2009, and December 31, 2011; and published in one of the selected pharmacy journals) were included in the data extraction process. During the data extraction process, abstracts were excluded from the final analysis if the study design was not consistent with that of an RCT or if an electronic PubMed abstract was not available.

Search strategy

The PubMed database was queried on October 24, 2012, by the lead investigator (Blair). Each journal was searched using the journal search function in the NLM catalog (e.g.,“Pharmacotherapy”[Journal]) combined with “Randomized Controlled Trial” as the publication type. Search results were then limited to abstracts published between January 1, 2009, and December 31, 2011. See the online only appendix for full search strategy.

Data extraction

An electronic data extraction form was developed using Adobe Acrobat Pro (version 10.1.4). This procedure was conducted for a number of reasons: first, to standardize the appearance of each abstract to decrease bias associated with reviewer preferences in formatting; and second, to allow the blinding of both the journal title and year of publication during the second phase of data extraction. Lastly, the data extracted from each abstract could easily be compiled without the risk of typographical errors associated with manual input of data into a spreadsheet.

Data extraction was carried out in two phases: phase 1 consisted of extracting the journal of publication, year of publication, article title, and body of each abstract that met the inclusion criteria. The extracted data were placed into individual electronic data extraction forms. The journal of publication and year of publication, once inputted into the form, were electronically blinded to the reviewer (Blair) before the second phase of data collection began. Unblinding of the journal and year of publication did not occur until all of the abstracts had been evaluated against the CONSORT recommended items for abstracts. The full-text article for each abstract was not reviewed or compared to the abstract during data extraction. The order in which the forms were to be scored during the second phase of data extraction was randomized. Randomization was achieved by assigning each abstract a randomization number using the math.random Javascript embedded in each electronic form. The reviewer (Blair) was instructed to begin with the abstract that was assigned the lowest randomized number value and to proceed in order of increasing magnitude until all abstracts had been reviewed.

The CONSORT explanatory document for abstracts 7 was thoroughly reviewed before and during the second phase of data extraction to ensure a consistent interpretation of each CONSORT item being evaluated. Each abstract was reviewed to identify which items recommended by the CONSORT checklist were adequately reported or not reported (Table 1). The data extraction form, produced in the first phase, contained a checklist of the recommended CONSORT items. For each item, the reviewer was able to indicate which CONSORT items were adequately reported in a given abstract.

Table 1.

CONsolidated Standards Of Reporting Trials (CONSORT) criteria for abstracts^* and overall compliance from study sample (n = 63)

Data analysis

Descriptive statistics were computed for the primary endpoint: median overall compliance of pharmacy journal abstracts to the CONSORT recommendations for abstracts (defined as the median number of items in the abstract that are recommended by the CONSORT guidelines divided by the sum of the maximum attainable CONSORT score of 16). Please refer to Table 1 for the 16 items recommended by CONSORT to be in an RCT abstract. The secondary endpoints were the median overall compliance by journal and year of publication. Individual descriptive statistics for each reported item were analyzed using Minitab version 16.2.2 and Excel 2010.

RESULTS

From January 1, 2009, to December 31, 2011, a total of sixty-seven RCT abstracts were identified in the five pharmacy journals. Four abstracts were excluded from the final analysis; two from Am J Health Syst Pharm for not being RCTs and two from Ann Pharmacother due to the lack of an electronic abstract or for reasons relating to absence of an RCT study design. In total, sixty-three abstracts were included in the final analysis.

Most of the RCTs during the period of analysis were published in Pharmacotherapy (41.0%) followed by Ann Pharmacother (30.2%), Am J Health Syst Pharm (20.6%), and J Am Pharm Assoc (6.3%); only 1 abstract was assessed from Consult Pharm. Of the 3 years of publications assessed, 31.7%, 33.3%, and 34.9% of the abstracts were from full-text publications published in 2009, 2010, and 2011, respectively.

None of the 63 abstracts included all of the 16 items recommended by the CONSORT guidelines for abstracts. The median number of items that were included for all abstracts assessed was 9 (interquartile range [IQR] 8 to 10).

The median overall compliance to CONSORT for all abstracts assessed was 56.0% (IQR 50.0%–62.5%). This result indicated that RCT abstracts published in pharmacy journals included only about half of the items recommended by the CONSORT guidelines to be in an RCT abstract. Furthermore, the median overall compliance did not differ between journal and/or year of publication.

Am J Health Syst Pharm and Pharmacotherapy both had the highest median overall compliance: 56.25% (IQR 50.0%–56.26%) and 56.25% (IQR 50.0%–62.5%), respectively. Ann Pharmacother and J Am Pharm Assoc were found to be less compliant: 50.0% (IQR 43.75%–49.5%) and 50.0% (IQR 43.75%–56.26%), respectively. The lowest median overall compliance score was for Consult Pharm; however, this is based on 1 abstract and should be interpreted with caution.

Regarding individual abstracts, the highest single compliance score (percentage of items recommended by CONSORT included in an individual abstract) was 81.25%. This abstract was published in a February 2009 issue of Ann Pharmacother. The lowest single compliance score of 31.25% occurred in 2 separate individual abstracts (one from Ann Pharmacother published in October 2010 and the other in Am J Health Syst Pharm published in March 2009).

In relation to individual CONSORT recommended items, all RCTs reported conclusions (100%) and most abstracts reported outcomes (97%), objectives (97%), and interventions (90%). Conversely, insufficient reporting of the randomization procedures, reporting of blinding procedures, and identification of the study as an RCT in the title were almost universal. Registration of RCTs and sources of funding were not reported in any of the abstracts.

DISCUSSION

This study indicates that RCT abstracts indexed in PubMed from five major pharmacy journals do not conform to the CONSORT guidelines for reporting abstracts. Reporting RCT data in the form of an abstract should be as clear and detailed as possible. Omission of valuable information (the prevailing issue in this study) can allow practitioners to make incorrect assumptions about the intention, design, quality, and results of a given study. This is especially important given that abstracts can and do often affect clinical decision making.

As can be viewed in Table 1, CONSORT items scoring low in the sample include: “title,” “randomization,” “blinding,” “recruitment,” “harms,” “trial registration,” and “funding.” All of these items in the CONSORT tool—except for “harms,” “trial registration,” and “funding”—are descriptive of the trial methodology or status, whereas “harms” represents undesired subject outcomes, and the “trial registration” and “funding” items are quality measures to assist in identifying various sorts of bias. In a clinical context, of all these items, the lack of reporting associated with subject harms is most concerning. In the current era of medical practice, much greater scrutiny is placed upon using potentially unnecessary drug therapies and weighing potential therapeutic benefits of a particular agent versus potential harm. Theoretically and based upon this sample, if only the abstract (and not the full-text article) is utilized to make a clinical decision in favor of use of a particular drug, practitioners have the potential to recommend use of an agent without knowing potential risks 66% of the time.

The results of this study closely mirror the results of other investigations, as many studies have evaluated the quality of RCT abstracts as determined by the inclusion of items recommended by the CONSORT extension for abstracts 10, 12, 17–20. Conventional wisdom would state that abstracts are mostly accurate; however, similar studies in other academic disciplines have concluded that abstracts do have deficiencies. One such study compared abstracts from high-impact medical journals to the corresponding full-text article and found numerous errors and/or omissions 5, which is consistent with more recent studies 12. Furthermore, numerous studies have assessed the quality or accuracy of different specialty journals outside the medical profession 11, 13–15, 19–22, including pharmacy journals 4.

All of the pharmacy journals included in this study offered guidance to authors for the information to be included in abstracts of articles submitted for publication. However, as of June 2013, none of the reviewed pharmacy journals explicitly recommended the CONSORT extension for abstracts in their instructions for authors. The pharmacy journals included in this current study did set word limits (ranging from 250 to 300) for abstracts, which can increase the difficulty of including all 16 recommended items. However, during the drafting of the CONSORT recommendations, the CONSORT group concluded that all 16 items could be addressed while still conforming to the word limits imposed by journals. Examples of abstracts using the CONSORT checklist for abstracts are available on the CONSORT website at http://www.consort-statement.org.

This current research project identified that the median overall compliance to CONSORT for the 5 selected pharmacy journals was 56.0%. This rate is higher than that reported in similar analyses in other fields of medicine. Abstracts from anesthesia journals averaged 29.0% 21, while Chinese medical journals, on average, reported only 3 items from the CONSORT checklist 14. However, an assessment of abstracts in leading Chinese medical journals of traditional Chinese medicine reported a much higher compliance rate of 55.0% 15. An analysis of orthodontic and dental specialty journals reported even higher mean overall reporting quality scores of 60.2% and 62.5%, respectively 11, 19.

Limitations

One limitation of this current study is the potential for intra-rater variability. Certain criteria that the CONSORT extension evaluates can be interpreted subjectively, and because the abstracts were graded over a short span of time, scores could have been influenced by changing definitions of individual criteria. Additionally, the PubMed searching strategy for RCTs used “randomized clinical trial” as a publication type only. This searching strategy could have overlooked some eligible RCTs to include, particularly RCTs that were incompletely indexed (e.g., [in process]) at the time of the initial search.

CONCLUSIONS

The results of this study indicate that RCT abstracts published in PubMed from five pharmacy journals do not conform to all the CONSORT recommendations. Approximately half of the recommended items from the checklist are being routinely included in these RCT abstracts. This is problematic because health care providers at times rely on abstracts. Pharmacy journals should require authors to follow the CONSORT recommendations for abstracts to prepare RCT abstracts with informative and complete content. Better reporting of the items proposed by the CONSORT recommendations for abstracts can assist readers in identifying RCTs that are relevant to their practice.