Electronic resources preferred by pediatric hospitalists for clinical care

Abstract

Objectives

There is little research on pediatric hospitalists' use of evidence-based resources. The aim of this study was to determine the electronic resources that pediatric hospitalists prefer.

Methods

Using a web-based survey, the authors determined hospitalists' preferred electronic resources, as well as their attitudes toward lifelong learning, practice, and experience characteristics.

Results

One hundred sixteen hospitalists completed the survey. The most preferred resource for general information, patient handouts, and treatment was UpToDate. Online search engines were ranked second for general information and patient handouts.

Conclusions

Pediatric hospitalists tend to utilize less rigorous electronic resources such as UpToDate and Google. These results can set a platform for discussing the quality of resources that pediatric hospitalists use.

Most physicians agree that clinical decisions should be supported by accurate, non-biased, and current scientific evidence [1]. This proves challenging for busy clinicians who strive to practice evidence-based medicine (EBM) but have limited time to keep up with the rapidly expanding amount of medical knowledge [2]. Until recently, it was commonly felt that physicians answered most of their clinical questions by consulting peers and referencing textbooks or peer-reviewed sources such as MEDLINE. A recent study, however, showed that 90% of physicians now use the Internet to answer clinical questions [3]. Physicians have access to a vast number of online search engines such as Google [4] and “pre-filtered” or “evidence-based” databases in which the evidence is already summarized [5]. These resources have the potential to help physicians overcome the most common obstacle to practicing EBM, which is lack of time [6].

The practice of EBM requires an understanding of the hierarchical system of classifying evidence, commonly referred to as the levels of evidence. A variety of EBM hierarchical models have been developed to rank and organize the myriad of resources that are available to clinicians [7–10]. Generally, the quality of evidence moves upward through synopses of studies, syntheses, synopses of syntheses, and summaries. The lowest level includes “background information/expert opinion” resources such as UpToDate™ [10].

In 2010, the Pediatric Hospital Medicine Core Competencies formally defined the standards for knowledge and skills that are expected of all pediatric hospitalists. The document states that hospitalists should “be proficient in an EBM literature search using electronic resources such as PubMed” [11]. Given the paucity of data in this area, the primary aim of the authors of this study was to determine the patterns of electronic resources preferred by pediatric hospitalists to support medical decisions. The secondary aim was to determine physician characteristics associated with reported use of higher-quality electronic resources.

METHODS

Data collection

In this cross-sectional survey, participants were recruited by email using the American Academy of Pediatrics (AAP) Section on Hospital Medicine (SOHM) electronic mailing list during June 2014. Subscription to the AAP SOHM electronic mailing list is available to practicing physicians who are members of the AAP SOHM, practitioners who spend at least 50% of their time as pediatric hospitalists, trainees interested in a career in hospital medicine, and other providers who are interested in practicing hospital medicine. At the time the survey was distributed, there were approximately 2,100 members, which represents the largest email listing of pediatric hospitalists [12]. The recruitment email included a brief description of the study and a link to the survey. A follow-up reminder email 2 weeks after the initial email was sent to the entire email discussion list, again requesting participation in the study. No incentives were offered for participation. Inclusion criteria included only current practicing pediatric hospitalists (self-reported). The study was approved by the institutional review board of the Children's National Health System.

Survey instrument

The thirty-two-item survey consisted of two sections and was developed from the current EBM literature (online only appendix) [13–18]. Participants reported the electronic resources that they preferred to use to support clinical care (general background information, treatment information, drug information, patient handouts) and number of resources they routinely used on a monthly basis. Participants were able to choose from a list of electronic resources that spanned all levels of Glover's EBM pyramid [10]. Physician characteristics that might influence preferred resources were also collected, including practice location, year of practice post residency, and orientation to lifelong learning. Lifelong learning was measured using the Revised Jefferson Scale of Physician Lifelong Learning, a fourteen-question survey previously validated among a wide variety of specialties including pediatrics [19–21]. All items are directly scored based on their Likert weight, and the higher the score, the more positive is the respondent's orientation toward lifelong learning. The survey was piloted among six pediatric hospitalists to assess construct validity prior to distribution to the email discussion list. The survey was administered using Research Electronic Data Capture (REDcap), a secure, web-based application designed to support data capture for research studies that utilizes encryption technology to ensure confidentiality [22].

Statistical analysis

De-identified data were imported into SPSS, version 22 (IBM). Descriptive statistics were utilized to calculate the frequency distributions of reported usage for each of the 4 clinical questions. Fisher's exact test was performed to compare physicians in each of the 3 different practice settings and their reported use of each of the resources for each of the 4 different clinical questions. A one-way, between-subjects analysis of variance (ANOVA) was conducted to compare the effect of the primary practice setting on the frequency of electronic resources used in the past month. The likelihood of using a given resource for each of the 4 different clinical questions based on lifelong learning score was estimated using logistic regression. In addition, logistic regression analysis was used to assess whether the number of years since graduating from residency predicted the preferred usage of a particular resource for each of the 4 different clinical questions. Univariate and multivariate linear regression models were used to determine if lifelong learning score and the number of years since graduating from residency predicts the total number of resources used each month to answer any clinical question.

RESULTS

Response rate and demographics

Overall, 116 pediatric hospitalists completed the survey (5% response rate). Table 1 (online only) shows a summary of the responses to the demographic questions from the survey.

Outcomes

For the main outcome measure, there was substantial overlap in preferred resources, with the only exception drug information–related questions (Table 2). The most cited resource for clinical questions regarding general information, treatment, and patient handouts was UpToDate. The use of Google, electronic journal websites, and MEDLINE also appeared prominently for general information and treatment-related questions. Fisher's exact test did not reveal any statistically significant relationships between practice setting and resource preference. Also, there were no statistically significant differences between groups in regard to reported number of EBM resources used per month for the 3 settings: university affiliated children's hospital, university affiliated community hospital, and non-university affiliated community hospital [F(3, 112) 0.357, P=0.784] (Figure 1, online only). After Bonferroni correction for multiple comparisons, there were no statistically significant differences among groups.

Table 2.

Summary of the top 5 electronic resources used to answer 4 categories of clinical questions by practice setting

DISCUSSION

The results indicate that pediatric hospitalists utilize UpToDate more often than electronic journal websites and MEDLINE to support clinical decisions. Google, an online search engine, was ranked second for general information questions and patient handouts. If UpToDate is consider to be lower quality, our results suggest that pediatric hospitalists routinely use lower-quality evidence-based resources to inform their clinical practice. Physician characteristics—such as years of experience, practice setting, and orientation to lifelong learning—were not associated with resource utilization. In addition, the preferred resources of community-based hospitalists did not seem to vary greatly from the resources used by hospitalists affiliated with children's hospitals.

Effect of lifelong learning

The authors hypothesized that lifelong learning score and years of experience might influence a participant's preference for resources. However, our findings paralleled the results of a prior study that also found that demographic data could not predict physicians' reported use of EBM resources [23]. It is possible that physicians with a higher lifelong learning score may not necessarily use higher-quality resources but may seek answers to their clinical questions on a more consistent basis or use non-electronic resources (i.e., textbooks or colleagues). Prior studies have reported that less-experienced physicians are more likely to use clinical guidelines than experienced ones [24]. In addition, medical students have been shown to prefer electronic resources to paper textbooks [25]. Yet our study did not find any association between years of experience and reported use of resources. Part of the explanation of this finding may be that the ease of use of many newer electronic resources allows physicians of all ages to readily adapt to them.

Our findings contrasted with the pattern noted in other studies in which traditional peer-reviewed resources, such as individual journals and/or MEDLINE, were the resources more commonly used by physicians [14, 26–29]. Only one prior study listed UpToDate as the preferred nonhuman resource to answer clinical questions [30].

Previous studies have identified characteristics of electronic resources that appeal to clinicians: concise, easy to access, attractively presented, easy to understand, and low cost [31–33]. Our results suggest that pediatric hospitalists may be choosing potentially biased secondary sources such as UpToDate or non-EBM search engines such as Google because they appear to be more efficient and convenient than higher-quality EBM resources. The pediatric hospitalists may also not be aware of the limitations of these resources. While Google can be a valuable resource that provides a gateway to high-quality, peer-reviewed material, it also can be the source of misinformation and non-peer-reviewed material. A recent study found that data returned by Google are often not evidence based and instead include advertisements, foreign media sources, and personal websites [4].

Newer resources

The original model of EBM, in which physicians meticulously dissect the methods and results section of individual journal articles, may no longer be practical given clinicians' workload and the volume of new information. A MEDLINE search, for example, can on average take up to twenty-seven minutes to complete [34]. A newer strategy to incorporate EBM into one's practice has evolved in which physicians access pre-filtered sources that summarize the literature and help them sort through the large volume of literature to efficiently address clinical questions. Although abundant and convenient, these newer resources can have substantial differences. Prorok and colleagues examined ten separate electronic resources and found they varied substantially in terms of the quality of evidence, breadth of content, and timeliness of updating [35]. Another group of researchers discovered similar results when they compared four commonly used online evidence-based textbooks. The variation in treatment recommendations was attributed to differences in the rate at which theses resources are updated [5]. Therefore, the task of finding the highest level of evidence to answer clinical questions is no longer straightforward given the abundance of resources that are available online.

Implications

Predictably, every resource, including UpToDate, has its strengths and weaknesses [36]. The authors of a recent study concluded that no single resource was ideal, and therefore, clinicians should not rely on a single point-of-care resource [35]. Our data showed that respondents on average checked 3.5 different resources for each of the 4 clinical categories. This further suggests that given the myriad of clinical questions that a physician may ask, answers may not be found using the first resource that a physician chooses. The most recent evidence may not always be available in a pre-filtered resource [37]. Therefore, physicians must have expertise in accessing a variety of EBM resources. In our opinion, the ability to critically appraise an individual article remains a useful skill that should continue to be taught along the medical education continuum. An additional focus of EBM curricula should include: (1) exposing physicians to higher-quality EBM resources such as systematic reviews, meta-analyses, and journals that house critically appraised individual articles and (2) enabling physicians to critically analyze web-based search tools. Pediatric hospitalists are not able to fully gain EBM competencies during their training years. The field, therefore, must provide opportunities such as the annual Pediatric Hospital Medicine Meeting to help hospitalists become proficient in EBM literature searches.

These results imply that, despite the myriad of EBM resources that are now available, pediatric hospitalists tend to utilize less rigorous and lower-level electronic resources, such as UpToDate and Google. These findings should provoke discussion of the value of the clinical information sources that pediatric hospitalists and other clinicians use.

Strengths and limitations

The main strength of this study is the novel population surveyed. Although several existing studies have explored the information-seeking behaviors of other health care professionals, this is the first to provide important information regarding the preferred electronic resources used by pediatric hospitalists, one of the fastest growing fields in pediatrics. Pediatric hospitalists are now responsible for the majority of trainee education during inpatient clinical rotations [38–41]. Understanding their EBM practices is important because they serve as role models for future physicians. Pediatric hospitalists have a responsibility to help learners develop sufficient critical appraisal skills by explicitly specifying the strength of evidence that guides their own clinical decisions. When a lack of high-level evidence exists, which is often the case in pediatrics, they have a duty to explain to their trainees that their decisions are based on expert opinion or anecdotal evidence.

This study has several limitations common to web-based surveys. First, the estimated response rate for this study was 5%. This means the survey results are considered exploratory, not definitive. While 5% might seem low for survey research, previous Internet surveys of physicians report that response rates under 20% are not uncommon [42–45]. Other studies utilizing the AAP SOHM email discussion list have also been limited by low response rates [46–49]. In addition, precise calculation of response rates is another challenge of web-based surveys because it is unknown how many subscribers are active members or have duplicate email accounts [50].

Second, sampling may bias our findings [51]. Demographic data of email discussion list members was not available, and therefore, it was not possible to determine if the SOHM email discussion list reflected the general population of pediatric hospitalists. However, 87% of our respondents were employed at a university-affiliated children's hospital, which was similar to the demographics of pediatric hospitalists from another recent study, in which 82% of respondents described their setting as “academic” [46]. Furthermore, due to the small number of responses from hospitalists who practice in community settings, the results of this study may not be generalizable to those settings. The survey was structured in a way to help facilitate recall (i.e., “In the last month, how many of the following resources have you used?”) [52]; however, given the variability in the clinical responsibilities from month to month of many pediatric hospitalists, this might have influenced the results.

Finally, there may be a recall bias because the preference and use of resources was self-reported rather than directly observed. This raises the possibility that respondents' responses might not reflect actual practice. If this was the case, one might expect the bias to be in favor of higher-quality electronic resources, which was still remarkably low.

Biography

Jimmy B. Beck, MD, MEd , Jimmy.Beck@Seattlechildrens.org, Adjunct Instructor of Pediatrics, Division of Hospital Medicine, Children's National Health System, 111 Michigan Avenue NW, Suite M-4800, Washington DC 20010*; Joel S. Tieder, MD, MPH, Joel.Tieder@Seattlechildrens.org, Assistant Professor of Pediatrics, Division of Hospital Medicine, Department of Pediatrics, Seattle Children's Hospital and University of Washington School of Medicine, M/S FA.2.115, P.O. 5371, Seattle, WA 98105