Evidence-Based Medicine and the Practicing Clinician

Abstract

OBJECTIVE

To assess the attitudes of practicing general internists toward evidence-based medicine (EBM—defined as the process of systematically finding, appraising, and using contemporaneous research findings as the basis for clinical decisions) and their perceived barriers to its use.

DESIGN

Cross-sectional, self-administered mail questionnaire conducted between June and October 1997.

SETTING

Canada.

PARTICIPANTS

Questionnaires were sent to all 521 physician members of the Canadian Society of Internal Medicine with Canadian mailing addresses; 296 (60%) of 495 eligible physicians responded. Exclusion of two incomplete surveys resulted in a final sample size of 294.

MAIN RESULTS

Mean age of respondents was 46 years, 80% were male, and 52% worked in large urban medical centers. Participants reported using EBM in their clinical practice always (33, 11%), often (173, 59%), sometimes (80, 27%), or rarely/never (8, 3%). There were no significant differences in demographics, training, or practice types or locales on univariate or multivariate analyses between those who reported using EBM often or always and those who did not. Both groups reported high usage of traditional (non-EBM) information sources: clinical experience (93%), review articles (73%), the opinion of colleagues (61%), and textbooks (45%). Only a minority used EBM-related information sources such as primary research studies (45%), clinical practice guidelines (27%), or Cochrane Collaboration Reviews (5%) on a regular basis. Barriers to the use of EBM cited by respondents included lack of relevant evidence (26%), newness of the concept (25%), impracticality for use in day-to-day practice (14%), and negative impact on traditional medical skills and “the art of medicine” (11%). Less than half of respondents were confident in basic skills of EBM such as conducting a literature search (46%) or evaluating the methodology of published studies (34%). However, respondents demonstrated a high level of interest in further education about these tasks.

CONCLUSIONS

The likelihood that physicians will incorporate EBM into their practice cannot be predicted by any demographic or practice-related factors. Even those physicians who are most enthusiastic about EBM rely more on traditional information sources than EBM-related sources. The most important barriers to increased use of EBM by practicing clinicians appear to be lack of knowledge and familiarity with the basic skills, rather than skepticism about the concept.

Evidence-based medicine (EBM) is “the process of systematically finding, appraising, and using contemporaneous research findings as the basis for clinical decisions.”¹ Although the philosophical underpinnings of the EBM movement are more than a century old,² the term was coined and its explicit incorporation into medical education and clinical practice has become widespread only in the past two decades. For instance, a cursory examination of medlinefor the textword “evidence-based medicine” reveals 0 citations from 1966 to 1991, 17 from 1992 to 1994, and 132 from 1994 to 1996. Moreover, as a result of some studies suggesting that physicians trained in EBM are more likely to keep up-to-date than their traditionally trained counterparts,^3,4 recent physician training reports have called for increased exposure to EBM at the undergraduate and postgraduate levels.⁵

The practice of EBM involves four primary steps: formulating a clear question based on a patient problem, identifying relevant studies from the literature, critically appraising the validity and usefulness of the identified studies, and applying the findings in clinical practice.⁶ Despite enthusiasm in the educational and research communities for EBM, the attitudes of practicing general internists about EBM (and their comfort with the basic skills needed to practice EBM) have not been systematically investigated. Although EBM encourages the use of primary research studies, evidence-based clinical practice guidelines, and systematic overviews to inform treatment decisions,⁷ recent surveys have suggested that most physicians still rely heavily on the opinion of colleagues or consultants when making these decisions.^8–10 Although various barriers to the application of EBM principles in clinical practice have been described,^6,11,12 their relative importance to the practicing clinician is unknown.

In this article, we report the results of a survey designed to determine the attitudes of Canadian general internists toward EBM, define which information sources significantly affect their clinical decision making, and delineate the barriers to the use of EBM in the practice setting.

METHODS

Survey Sample

To learn the attitudes of general internists about EBM, we mailed a self-administered questionnaire in June 1997 to all 521 physician members of the Canadian Society of Internal Medicine who were currently residing in Canada (excluding the authors). The Canadian Society of Internal Medicine, a voluntary organization, is the only national advocacy organization for general internists in Canada. Second and third mailings were carried out in August and September 1997 for nonrespondents. The study was closed in December 1997. The major inclusion criterion for this survey was that the respondent be involved in active patient care to some extent. Thus, retired physicians and those who reported that they spent all of their time in administration or research were ineligible for the study.

Measurements

The study questionnaire was developed after review of previously published questionnaires on clinical practice guidelines.^8,10,13 Pilot testing was done with five general internists to determine the acceptability and clarity of the questionnaire and to confirm its face validity.

In addition to demographic data, the survey included questions assessing the respondents’ self–reported use of EBM in clinical practice (always, often, sometimes, rarely, or never), their opinions about EBM and the barriers to its use, and their use of various information sources. Attitudes were assessed using 5-point ordinal scales with anchors specific to the question (e.g., 1 = never use, 5 = always use; 1 = strongly disagree, 5 = strongly agree). For the purposes of this survey, EBM was defined as “the process of systematically finding, appraising, and using contemporaneous research findings as the basis for clinical decisions.”¹

We sought evidence of the validity of respondents’ reports of use of EBM by looking for expected associations between their reports of use of EBM and other variables. For example, self-reported use of EBM was highly associated with reported use of primary research articles ( p = .006) and self-confidence in critical appraisal ( p = .01).

Data Analysis

Analyses determined whether self-reported use of EBM was associated with demographic or professional characteristics, attitudes toward EBM and possible barriers to its use, self-reported use of different information sources, and self-confidence in basic EBM skills. Differences in categorical variables were tested using Fisher’s Exact Test or the χ²test, with the modification for linear trend when appropriate. Differences in continuous variables were tested using the Student’s t test. All tests were two-sided.

To simplify the presentation of results, self-reported use of EBM in practice was transformed to a dichotomous variable classifying subjects who often or always used EBM as “EBM users” and respondents who sometimes, rarely, or never used EBM as “EBM nonusers.” Analyses were conducted both with the dichotomous variable and with the original ordinal variable; results were similar, so only results for the dichotomous variable (EBM use and EBM nonuse) are reported.

RESULTS

Description of Respondents

Of the 521 questionnaires distributed, 26 (5%) were ruled ineligible because the physicians had died (1), moved without forwarding address (2), retired from active practice (11), or reported that they did not spend any time in direct patient care (12). Of the 495 eligible questionnaires, 296 (60%) were returned to the study coordinating center; 2 incomplete surveys were excluded from further analysis, and the final sample thus consisted of 294 completed surveys.

The demographic characteristics of respondents were similar to those of all members of the Canadian Society of Internal Medicine, in mean age (46 years), gender (80% male), and location of practice (52% in large urban centers) (data not shown). Approximately 20% of Canadian physicians practicing as general internists are members of the Canadian Society of Internal Medicine (1995 Royal College of Physicians and Surgeons of Canada Work Force Study, unpublished data). Although members of the Canadian Society of Internal Medicine have a similar age distribution as nonmembers, members are more likely to be female (20% vs 12%, p < .0001), have academic appointments (53% vs 39%, p < .0001), and have teaching responsibilities (62% vs 48%, p < .0001). Conversely, nonmembers are more likely to spend more than 20 hours per week in direct patient care (91% vs 84%, p < .0001). When asked how often they employed EBM in their clinical practice, 33 (11%) of the 294 respondents stated “always,” 173 (59%) “often,” 80 (27%) “sometimes,” and 8 (3%) “rarely/never.”

In comparing the demographic characteristics of EBM users with EBM nonusers, no significant differences were detected on univariate analysis (Table 1) or multivariate analysis (data not shown). In particular, those respondents with academic appointments were no more likely to state that they use EBM than respondents without academic appointments (70% vs 71%, p = .80). Moreover, there were no significant differences between the 33 physicians who reported that they “always” used EBM and the other 261 respondents for those items listed in Table 1 (data not shown).

Table 1.

Demographic Characteristics of Respondents, by Self-reported Frequency of Use ofEvidence-Based Medicine (EBM) Principles in Clinical Practice

Characteristic	EBM Users*(n = 206)	EBM Nonusers*(n = 88)	p value
Age (mean ± SD)	45.1 ± 11.1	47.1 ± 11.2	.25
Female sex, n(%)	42 (20)	18 (20)	.98
Year of graduation, n(%)			.35
After 1990	42 (20)	11 (13)
1980–89	55 (27)	24 (27)
1970–79	53 (26)	29 (33)
Before 1970	56 (27)	24 (27)
Years of postgraduate training (mean ± SD)	5.1 ± 1.3	5.2 ± 1.3	.48
Self-reported type of practice, n(%)			.12
Solo practice	77 (37)	44 (50)
Group practice	89 (43)	32 (36)
Other	40 (20)	12 (14)
Location of practice, n(%)			.21
Urban >250,000	113 (55)	40 (45)
Urban 50,000 to 250,000	52 (25)	33 (38)
Urban < 50,000	24 (12)	9 (10)
Rural	17 (8)	6 (7)
Hours per week spent in direct patient care, n(%)			.67
<20	31 (15)	15 (17)
≥20	175 (85)	73 (83)
Have academic appointment, n(%)	110 (53)	48 (53)	.86
Have teaching responsibilities, n(%)	132 (64)	50 (57)	.24
Regularly use computer, n(%)	174 (84)	69 (78)	.21
Initiate or publish research studies, n(%)			.17
≤2 in past 2 years	178 (86)	81 (92)
≥3 in past 2 years	28 (14)	7 (8)

^*EBM users are those physicians who reported often or always using EBM; EBM nonusers are those physicians who reported only infrequently or never using EBM.

Attitudes Toward Evidence-Based Medicine

Self-identified EBM users more often agreed with positive statements about EBM and its impact on patient outcomes and cost-effectiveness of care than did EBM nonusers. However, the vast majority of both groups felt that EBM helps clinical decision making and that physicians should be able to critically appraise the literature. Although EBM nonusers were more likely to feel that EBM was the purview of academics, only a minority of respondents agreed with statements of possible barriers to the practice of EBM (Table 2).

Table 2.

Attitude Toward Evidence-Based Medicine (EBM)

	Agreement*
Attitude	EBM Users(n = 206),n(%)	EBMNonusers(n = 88),n(%)	Overall Degreeof Agreement Mean ± SD
Positive statements about EBM
EBM can play a positive role in clinical practice†	202 (98)	75 (85)	4.3 ± 0.6
Physicians must be able to distinguish methodologically sound frompoor research	192 (93)	80 (91)	4.4 ± 0.7
EBM helps clinical decision making‡	186 (90)	67 (76)	4.0 ± 0.7
EBM improves patient outcomes‡	127 (62)	37 (42)	3.6 ± 0.8
Clinical decisions should be based on the best numerical estimates ofrisks and benefits	114 (55)	40 (45)	3.4 ± 0.9
EBM leads to more cost-effective practice	101 (49)	34 (39)	3.4 ± 0.8
Possible barriers to the practice of EBM
Proponents of EBM tend to be academics rather than front-line clinicians†	83 (40)	54 (61)	3.2 ± 1.0
In most areas of medicine, there is little or no evidence to guide practice	53 (26)	24 (27)	2.7 ± 1.0
EBM is a new concept	54 (26)	20 (23)	2.8 ± 1.0
EBM devalues clinical experience and intuition	30 (15)	20 (23)	2.5 ± 1.0
EBM is impractical for everyday clinical practice‡	22 (11)	20 (23)	2.5 ± 0.9
EBM removes the “art” from medicine‡	18 (9)	18 (20)	2.2 ± 1.0
EBM de-emphasizes history taking and physical examination skills†	14 (7)	18 (20)	2.1 ± 1.0

^*Agreement was defined as the proportion of respondents choosing 4 (agree) or 5 (strongly agree) for each statement. Overall degree of agreement was ascertained using a 5-point Likert scale with 5 = strongly agree and 1 = strongly disagree.

^†p < .001 for comparison between EBM users and nonusers.

^‡p < .01 for comparison between EBM users and nonusers.

Preferred Sources of Information

Despite the general enthusiasm for EBM, only a minority of respondents reported frequently using evidence-based clinical practice guidelines or studies identified from a focused literature search to guide their clinical decisions (although EBM users were significantly more likely to use these resources than EBM nonusers). Clinical experience, review articles, and the opinion of colleagues were the favored sources of information for both groups of physicians (Fig. 1). Information sources that only a few respondents stated they “frequently used” included the Internet (10%), Cochrane Collaboration Reviews (5%), and pharmaceutical industry representatives (2%).

FIGURE 1.

Use of information sources to guide clinical decision making. Results are expressed as percentage of respondents who reported using the information sources “often” or “always.” Research articles defined as “articles from focused searching of electronic databases (e.g., medline).” The differences between evidence-based medicine (EBM) users and nonusers were significant only for use of clinical practice guidelines (p = .01) and use of research articles ( p = .0001).

The median number of professional journals regularly read by respondents was 4 (no difference between EBM users and nonusers, p = .32). While 214 (73%) reported reading professional literature for more than 1 hour per week, only 46 (16%) read for more than 4 hours per week; after excluding the 40 physicians who were currently pursuing further postgraduate training, only 14% of physicians practicing “full-time” read for more than 4 hours per week. Reported reading times were greater ( p = .015) for EBM users than nonusers; for example, 19% of EBM users reported reading professional literature more than 4 hours per week compared with 7% of nonusers.

Confidence in Evidence-Based Medicine Skills

While most respondents felt comfortable in their ability to formulate a clear question based on a specific patient problem, less than half felt confident in the other basic skills of EBM (Fig. 2). In particular, respondents were least comfortable with critical appraisal of identified studies. However, 93% felt that physicians must be able to critically appraise the literature (Table 2), and there was a high level of interest in task-specific training (56% requested a continuing medical education event on critical appraisal). Not surprisingly, EBM users expressed more confidence in literature searching and critical appraisal (both p < .01).

FIGURE 2.

Percentage of respondents expressing confidence in basic evidence-based medicine (EBM) skills. *p < .01 for comparison between EBM users and nonusers.

DISCUSSION

In summary, although these internists were generally positive about EBM and its role in clinical practice, respondents reported relying on their personal experience and the opinion of colleagues most often in their decision making. This finding is not new,^8,9,10,14 and is consistent with empirical evidence from the medical education literature,¹⁵ but the degree of reliance on these traditional sources of medical information is surprising given the high proportion of respondents who felt they “always or often” used EBM to guide their clinical decisions. However, as pointed out by others, experts and colleagues are a “quick, cheap, and easy to use” source of information and “also provide guidance, support, affirmation, and other psychological benefits that computerized sources cannot provide.”^14,16 These physicians reported using textbooks or review articles as a basis for clinical decisions more often than such evidence-based tools as clinical practice guidelines, Cochrane Collaboration Reviews, or primary research studies identified from focused literature searching. Although textbooks and review articles can be an excellent source of background information (for such things as identifying the presenting features of a disorder), they have limited utility for treatment decisions. The information in textbooks may be more than a decade out of date at the time of publishing,^17,18 and there is significant potential for unrecognized bias in review articles.^16,19 As pointed out by Smith,¹⁴ studies of physician needs for information consistently reveal that most questions arising from clinical encounters concern therapy.

The likelihood that physicians would incorporate EBM into their practice was not associated with any demographic or practice-related factors. The major barriers to the practice of EBM appear to be lack of knowledge and lack of the basic skills of EBM (particularly literature searching and critical appraisal of study methodology) rather than negative attitudes toward the concept. Echoing the results of earlier surveys with other physician groups,^20–22 most of the respondents who were uncomfortable with these tasks wanted to learn more about them. The fact that the major criticism of EBM appears to be the perception that its proponents are academics rather than front-line clinicians suggests that future educational strategies to enhance the use of EBM should target the front-line clinician.

Methodologic Considerations

This study has a number of limitations that must be acknowledged. First, this is a survey of physician attitudes, not an audit of actual practice, and thus the division of respondents into EBM users and EBM nonusers is based solely on self-report and therefore potentially inaccurate. As other studies have shown poor correlations between attitudes about clinical practice guidelines and compliance with the guideline recommendations,²³ future comparisons of EBM users and nonusers should start with more objective definitions (perhaps using practice audits to group physicians by their use of proven efficacious treatments). Nonetheless, our results provide indirect evidence of the validity of the respondents’ self-report of EBM use.

A second limitation of our study, as with any survey, is the possibility of social-response bias in the respondents’ answers. However, Covell et al., in a study comparing self-reported use and observed use of information sources, have shown that physicians tend to overreport their use of journals and textbooks and underreport their reliance on “human sources.”²⁴ Thus, our finding of a high self-reported reliance on colleagues and expert opinion appears robust and may even be an underestimate of the picture in actual practice.

Third, the response rate was 60%, and thus the external validity of the findings may be questioned. However, respondents were similar demographically to the total population of the Canadian Society of Internal Medicine, and the absence of significant differences in reported EBM use between those physicians responding to the first, second, or third mailings argues against a marked participation bias. Moreover, while members of the Canadian Society of Internal Medicine are more likely to be female and have academic teaching appointments than nonmembers, these factors did not predict stated EBM use. Thus, the results may be generalizable to other Canadian general internists.

In conclusion, practicing general internists reponding to this survery were enthusiastic about EBM and expressed a strong desire to learn more about the basic skills of EBM, particularly literature searching and critical appraisal. The major barriers to more widespread use of EBM appear to be lack of knowledge and experience and a perception that EBM is the purview of academics rather than practicing clinicians. The time appears ripe for an educational program targeted at general internists to enhance their knowledge and use of EBM in everyday practice.