Randomized Controlled Trial of RSS Reader Use and Resident Familiarity With Primary Literature

Abstract

Background

Awareness of the primary literature is important for clinicians. Lack of time, poor access to information, and lack of personal initiative may be barriers for some trainees. Really Simple Syndication (RSS) readers aggregate web content, such as journal abstracts, in a single location for easy viewing.

Objective

We assessed whether use of an RSS reader would increase resident reading frequency, familiarity, and understanding of the primary literature.

Methods

We conducted a single-center, prospective, randomized, nonblinded, controlled trial of the effect of RSS reader use on knowledge of recent literature among pediatrics residents. Residents were randomly assigned to the RSS group (education in RSS use and receipt of the Pediatrics RSS feed) or a control group that followed standard reading practices. Outcome measures were differences on baseline and monthly surveys of reading frequency, familiarity with recent publications, and knowledge of recent articles (familiarity validation).

Results

Of 144 eligible residents, 79 (55%) were enrolled in the survey, with 81% (64 of 79) of participants completing all surveys. The RSS reader use was correlated with greater familiarity with selected articles, but not with improved understanding (as measured by ability to answer multiple-choice questions about content). Participants reported satisfaction with the RSS reader based on its ease of use, accessibility, and as an aid in supplementing reading.

Conclusions

The RSS reader use was accepted by residents and associated with increased familiarity with the primary literature but not with increased understanding.

Introduction

Accessing and retaining the primary medical literature can be challenging for residents. Using technology to locate and assimilate evidence may help optimize learning.¹ Residents cite lack of time, poor access to medical information, and lack of personal initiative as barriers to more extensive reading of original research.^2–4 Residents use online resources to aid in the education process.^5,6 Information aggregation resources may help residents supplement core readings and improve familiarity of primary literature by making reading more efficient, information more accessible, and learning more organized.

Publishers make online content available via a web technology known as RSS, or Really Simple Syndication. The RSS readers are programs or web applications that aggregate RSS content—RSS “feeds”—in a single location for easy viewing. One subscribes to an RSS feed by entering the website address into the reader, which regularly checks the user's subscribed feeds for new articles, downloads updates, and provides a user interface to monitor, read, and save feeds.⁷ The RSS readers are accessible on all computers, smart phones, and tablets, bringing new content to the user in 1 clear interface (example reader: http://feedly.com/index.html#discover), rather than users needing to visit numerous websites for information. The RSS feeds are used to make medical content more accessible^8–11 and help medical professionals remain informed.^12,13 We are unaware of any studies investigating the effect of RSS readers on resident education.

We conducted a pilot study to investigate whether training pediatrics residents in RSS reader use improved their familiarity with primary pediatric literature. We hypothesized that residents trained in RSS reader use would be more familiar with recent articles compared with a control group, as measured by survey responses.

Methods

Sampling

We conducted a prospective, randomized, controlled, nonblinded study at the Children's Hospital of Philadelphia (CHOP). The pediatrics residency program is an accredited, university-based training program with 144 residents. Participants were randomized to the intervention group (RSS reader users) or the control group (residents who continued to access and read primary literature per their usual practice).

Intervention

The intervention group used Google Reader, which was selected for ease of use and compatibility with most web browsers and operating systems.¹⁴ Participants who used RSS readers before the study were asked to stop using their reader. The intervention group was e-mailed information about Google Reader, as well as how to create an account and subscribe to the Pediatrics RSS feed. Pediatrics was chosen because all residents received a print subscription.

Data Collection

Participants completed a baseline survey and 3 monthly surveys testing their familiarity with recent primary articles. The baseline survey included participants reading of primary literature, familiarity with recent articles, familiarity validation, and current use of computers, RSS readers, and additional educational resources. Monthly surveys contained questions about the frequency of reading primary literature, familiarity with recent publications, and familiarity validation with multiple-choice questions testing knowledge of recent articles (created by study designers). For monthly survey questions, 6 original research articles were selected from Pediatrics. An exit survey included qualitative questions about the satisfaction with RSS reader use, usability, desire to continue using RSS, and general feedback. Incentives ($5 and $10 gift cards) were offered for survey completion.

This study was deemed exempt from review by the Committee for the Protection of Human Subjects Institutional Review Board at CHOP.

Data Analysis

We used χ² analysis and Student t tests to compare baseline characteristics and monthly survey results between groups, respectively. Analysis was conducted on an “intention-to-treat” basis. Nonresponder data were coded as the lowest possible score (eg, for a familiarity question, a nonresponder was coded as “not familiar”) to mitigate bias from study attrition.

Results

Seventy-nine out of 144 eligible residents (55%) enrolled. Eighty-one percent (64 of 79) of the study participants completed the study. At baseline, there were no statistically significant differences between groups in any measure, including sex, class year, journal reading frequency, Pediatrics reading, selected article familiarity, previous RSS use, or resources used to answer medical questions (table 1). In addition, there was no difference between groups in individual assessment of technological “savvy” (level of agreement with the following statement: “I consider myself to be tech savvy, meaning comfortable using computers, various handheld devices, and searching for online content”).

TABLE 1.

Baseline Characteristics of Study Participants by Group

The intervention group showed greater familiarity with selected articles but did not show improved understanding, as measured by answering specific questions about the articles (table 2). Randomization to the intervention group was not associated with a change in reading frequency (reading frequency by group per month: month 1, RSS 1.2 versus control 0.79 [P  =  .03]; month 2, RSS 0.83 versus control 0.77 [P  =  .78]; month 3, RSS 0.73 versus control 0.9 [P  =  .36]).

TABLE 2.

Participants' Familiarity and Validation of Familiarity With Selected Articles

In the exit survey, participants reported overall satisfaction with RSS reader use to access primary literature. All 30 participants in the RSS reader group (100%) felt the reader was easy to set up and use. Eighty percent (24 of 30) of the RSS group commented RSS readers were an efficient way of remaining appraised of new articles and that they would continue to use readers for that purpose. Fifty-three percent (18 of 34) of the control group expressed a desire to be trained in RSS use because several control group participants had heard from the RSS group participants that the reader was useful. The RSS reader criticisms included difficulty accessing full content of articles (which requires authentication via an institutional proxy or journal website login) and finding time to read. During months with heavy clinical loads, both groups (RSS, 20% [6 of 30]; control, 32% [11 of 34]) reported limiting their reading to only targeted, patient care–related topics.

Discussion

In this randomized, controlled, nonblinded trial of RSS reader use by pediatrics residents, RSS reader use did not improve the depth of knowledge pertaining to primary literature articles. The “push” aspect of RSS readers may not provide an adequate form of education; RSS reader accessibility may not be enough to overcome time constraints, and the large volume of articles distributed may overwhelm trainees. Residents likely learn best when they actively choose what to read and apply that knowledge to clinical situations.

However, RSS reader use was associated with improved awareness of the primary literature. The modality was easily used by residents. Despite RSS reader use not increasing reading frequency or information recall, it may improve efficiency. Residents' awareness of articles may make them more likely to refer to those articles when confronted with specific clinical situations. Residents' use of aggregated articles provided by program leadership, through websites^15,16 or USB (universal serial bus) drives,¹⁷ and e-mail alert use increased awareness of primary literature.¹⁸ The RSS readers may provide residents with a means of reviewing the literature and remaining aware of recent information while extending their breadth of knowledge. Although Google ended their RSS reader program in July 2013, there remains demand for that service. Multiple companies have developed alternative RSS readers.¹⁹ At least 3 million people are subscribers on 1 free program, and paid programs are in high demand.²⁰

It is not clear whether increased familiarity or knowledge of articles affects clinical care. For e-mail–based research summaries, the evidence is limited; e-mail alerts were associated with self-reported practice improvement in 1 study,²¹ but had no effect on clinical practice in other studies.^22,23

Our study has several limitations. It is a small, nonblinded study of residents at 1 institution. Results may not be generalizable to trainees at other institutions. Second, although there was no difference between groups at baseline in previous RSS reader use, control group participants may have used an RSS reader during the study. Third, the content of the multiple-choice questions did not have evidence to support its validity. Finally, only the Pediatrics RSS feed was used. Each month, 100 citations related to that month's issue are released. The RSS reader group participants may have experienced data overload, discouraging their continued reading.

Conclusion

The RSS reader use did not improve depth of knowledge pertaining to primary literature articles but may improve awareness of current articles. Some limitations in the current approach, specifically the large citation volume on the Pediatrics RSS feed, could be overcome by developing customized RSS content “bundles” for distribution to trainees. This study showed that RSS readers are easy to use and accepted by residents. There may be merit to evaluating the delivery of curated, timely, and relevant journal citations in 1 interface accessible on any Internet-connected device.