Change in Physician Knowledge and Attitudes After Implementation of a Pneumonia Practice Guideline

Abstract

OBJECTIVE

To assess physicians' response to implementation of an emergency department (ED) pneumonia practice guideline and determine if the guideline changed physicians' knowledge and attitudes about pneumonia care.

DESIGN

Prospective intervention study with cross-sectional and longitudinal physician surveys.

SETTING

An urban, university teaching hospital ED.

PARTICIPANTS

One hundred forty physicians who were responsible for the triage of at least one of 166 patients presenting to the ED with community-acquired pneumonia.

MEASUREMENTS

Physician characteristics, attitudes about pneumonia care and guidelines, and ratings of guideline helpfulness and effects on patient care were obtained by self administered questionnaire before, during, and after a yearlong intervention.

MAIN RESULTS

More than 73% of the physicians reported the guideline as helpful and more than 94% wanted it to be continued in the future. Most reported that the guideline would decrease costs and improve quality without any increase in adverse outcomes. Two thirds said they were more likely to treat patients with pneumonia as outpatients in the future because of the guideline. Among the 58 physicians with matching preintervention and postintervention survey data, the guideline decreased the beliefs that “all patients>65 years old with pneumonia should be admitted,” from 52% to 14% (p < .001), and that “patients with pneumonia have a>15% mortality rate,” from 11% to 5% (p < .007). The intervention did not significantly change general attitudes about practice guidelines. House officers rated the guideline as more helpful than attending physicians (p < .02).

CONCLUSIONS

This locally developed, actively implemented guideline was well regarded by physicians. Guidelines can change practice and also alter underlying knowledge and attitudes about disease management. They may be most useful to those with less experience.

There has been great interest in the development and implementation of clinical practice guidelines as a means of improving quality, decreasing costs, reducing variation, and fostering evidence-based decision making. However, numerous studies have documented that practice guidelines do not always guide practice.^1–8 Physicians' acceptance of practice guidelines and perception of them as useful decision-making aids may be important determinants of their ultimate success. Prior work indicates that physicians express both positive and negative general attitudes about guidelines. Most physicians agree that guidelines are a convenient source of advice, good educational tools, and likely to improve quality of care.^9–12 At the same time, many express concerns about “cookbook” medicine, clinical rigidity, their loss of autonomy, and the emphasis on cost containment.^9–12

Previous studies of guidelines have predominantly focused on attitudes about guidelines in general. Physicians' acceptance of and response to specific guidelines currently in use have been less frequently assessed, but may provide important insights into why some guidelines succeed and others fail. Many efforts to change physician behavior are successful during the course of the intervention, but once the study is over, decision making reverts back to baseline practice. To achieve sustainable results, guidelines should do more than change physician behavior temporarily or coercively, they should change underlying knowledge, attitudes, and beliefs, thus changing future practice style.

We developed a series of surveys to measure physician response to an actively implemented guideline to reduce hospitalizations for patients presenting to the emergency department with community-acquired pneumonia. Our goals were to: understand physicians' knowledge, acceptance, and perceived usefulness of the guideline and its effects on patient care; determine if the guideline changed physicians' knowledge and attitudes about pneumonia care; and, examine whether experience with a specific guideline would influence general attitudes about guidelines.

METHODS

This investigation was part of a prospective interventional trial of an actively implemented guideline to safely decrease hospitalizations for patients presenting with pneumonia to the Emergency Department (ED) of the Massachusetts General Hospital, a large, urban teaching hospital in Boston, Mass. The hospital has approximately 63,000 ED visits and 800 pneumonia admissions each year. The medical section of the ED is staffed with 20 attending physicians and 138 house officers in internal medicine. The study has been previously described in detail.¹³

Study Patients

To be considered eligible for the guideline, patients had to have community-acquired pneumonia confirmed by chest x-ray findings and be 18 to 84 years old. Patients who would not be appropriate candidates for outpatient therapy were excluded (i.e., those with room air oxygen saturation < 90%, chronic oxygen dependence, inability to take oral medications, HIV disease, immunosuppression, recent hospitalization, nursing home residence, injection drug use, neuromuscular disease, pregnancy, psychiatric disease, substance abuse, homelessness, or lack of a telephone). A validated prediction rule, the Pneumonia Severity Index (PSI), was used to risk-stratify patients.¹⁴ The PSI is based on patient age, gender, selected comorbid conditions, and vital sign and laboratory values at presentation. Eligible patients with a PSI score of 90 or lower (30-day mortality risk < 2.8%) were considered “low risk,” and targeted by the guideline as appropriate candidates for outpatient therapy. The guideline instructed physicians to override the recommendation for outpatient treatment if clinical judgment indicated inpatient care was preferable. There were 166 eligible low-risk cases during the intervention period.

Guideline Development and Implementation

The guideline was developed locally by a multidisciplinary group that included experts in pneumonia outcomes research and opinion leaders in infectious diseases, pulmonary medicine, general internal medicine, and emergency medicine. This was one of a few guideline projects being implemented in the ED at the time. The guideline was disseminated to ED physicians (attending physicians and house officers) with face-to-face educational sessions. In addition, written materials were sent to all ED attending physicians, internal medicine housestaff, and primary care physicians affiliated with the hospital. The guideline was actively implemented in the ED by a dedicated study nurse who screened patients presenting with pneumonia, identified low-risk patients according to the guideline, and informed ED physicians of the patient's low-risk status. To facilitate outpatient therapy, patients could be sent home with a course of oral clarithromycin (Biaxin at 500 mg twice a day for 10 days), receive home nurse visits 24 and 48 hours after discharge (if desired by the physician), and have a follow-up outpatient physician visit arranged for them. The antibiotic was provided at no cost to the patient. Visiting nurse services were ordered for 27% of outpatients.

The study nurse was available weekdays from 8 amto 6 pm. At night and on weekends, the ED physician was responsible for guideline implementation. All decisions about appropriateness for outpatient therapy and choice of antibiotic agent were made by the ED physicians in consultation with the patient's primary care doctor (if available). The study was approved by the MGH Subcommittee on Human Studies. The guideline was implemented from April 1, 1996, through February 28, 1997.

Physician Survey Data

We surveyed all physicians before implementation began and after a year's experience. Prior to implementation, we asked all attending physicians (n = 22) and house officers (n = 138) who staffed the ED to complete a 4-page questionnaire. This questionnaire included questions about demographics, training, experience treating pneumonia, practice style (estimated percentage of pneumonia patients admitted to the hospital), and attitudes about pneumonia care. We used items from a previously validated instrument to assess general attitudes about practice guidelines.⁹ The postintervention survey repeated the questions about helpfulness of the guideline components, desire for future use, and the overall effect of the protocol on future admission decisions. It also contained two pneumonia attitude questions and five general guideline attitude questions. We sent up to two reminder notices with replacement questionnaires to all nonrespondents, if necessary.

Physicians were also surveyed after each patient encounter once the decision to admit or discharge the patient was made. The encounter survey assessed familiarity with the guideline and elicited ratings of the helpfulness of the protocol's components and desire for them to be continued in the future. Respondents were also asked to estimate the effect of the intervention on quality, cost, hospitalization rates, malpractice risk, patient satisfaction, adverse events, ease of decision making, and how their experience might change their threshold for admitting patients with pneumonia in the future. We did not distribute an encounter survey for 27 of the 166 cases. In these cases, the ED physician had no documented interaction with the protocol because the episode occurred when the study nurse was off duty (nights and weekends) and the ED physician failed to complete the guideline algorithm.

Statistical Analyses

We report values as mean ± SD or median and interquartile range (IQR) as appropriate. Likert scale responses were collapsed into categories of agreement (4 = somewhat and 5 = strongly agree), no opinion (3), and disagreement (1 = strongly disagree and 2 = somewhat disagree). For the subset of physicians who completed both preintervention and postintervention surveys, we created a difference score (between the two surveys) using each physician as his or her own control to assess whether the guideline changed attitudes using a paired student's t test. We used χ², Fisher's exact, and Wilcoxon tests to evaluate differences between respondents and nonrespondents, and attending physicians and house officers. Two-sided p values ≤ .05 were considered statistically significant. All analyses were performed with SAS 6.12 statistical software (Cary, NC).

RESULTS

Preguideline Survey

Physician Characteristics

We received a completed preguideline survey from 140 of 158 ED physicians (89% response rate). Eighteen (82%) of the attending physicians responded and 122 (88%) of the house officers (Table 1) Attending physicians were in practice for an average of 11.6 years, and more than half of them had completed formal emergency medicine residencies. Ninety-two percent of the house officers were internal medicine residents; the remaining 8% were medicine-pediatrics residents. Attending physicians evaluated a median of 5 study patients over the 1-year implementation period. Residents, who were only in the ED for an average of 1 month, saw a median of 1 study patient over the same period.

Table 1.

Characteristics of Respondents to the Preguideline and Postguideline Surveys

Physician Characteristic	Completed PreguidelineSurvey (n= 140)	Completed Pre- and PostguidelineSurvey (n= 58)	p Value *
Attending physicians
Number	18	15	.001
Years in practice, mean (SD)	11.6 (7.1)	12.0 (7.3)	>.2
Emergency medicine residency	10	10	<.2
House officers, n	122	43	.001
Interns	59	29
Residents	63	14
Female gender, n(%)	44 (31)	15 (26)	<.2

^*Comparing physicians who completed both surveys and those who only completed the preguideline survey. A greater percentage of attending physicians completed both surveys compared with house officers.

Pneumonia Experience, Knowledge, and Attitudes

Physicians estimated that they evaluated a median of 20 patients with community-acquired pneumonia in the preceding 12 months (IQR, 10, 40) and admitted a median of 60% of these cases (IQR, 40, 80). Overall, physicians' median threshold 30-day mortality risk necessitating hospitalization was 5% (IQR, 5%, 10%). At baseline, 65% of respondents agreed that “most patients would prefer to be treated as outpatients,” and only 19% felt that “most pneumonia patients need to be hospitalized.” However, there was considerable difference of opinion about whether “all patients >65 years old with pneumonia should be hospitalized” (52% agreed vs 47% disagreed).

General Attitudes About Guidelines

Physicians expressed both positive and negative general attitudes about practice guidelines. For example, approximately 80% of respondents felt practice guidelines were “good educational tools” and “a convenient source of advice.” At the same time, half of them said they were “oversimplified or cookbook medicine.”

Encounter Survey

We received 172 completed encounter surveys from which we assessed physicians' response to direct interaction with the guideline (often a resident and an attending physician assessed each patient). We received at least one completed encounter survey for 135 (97%) of the 139 cases in which the ED physician interacted with the study nurse or the guideline materials.

Knowledge and Acceptance of the Guideline

Overall, 68% of respondents (78% of attending physicians and 60% of residents) said they knew “a fair amount or great deal” about the guideline before being approached by the study nurse about a specific eligible patient. None of the attending physicians and only 10% of the residents said they knew “nothing at all” about the pneumonia guideline. Eighty-three percent of physicians accurately recalled the mortality risk stated by the guideline. They rated their patients' mortality risk as the same as the guideline prediction in 52% of cases, lower in 20%, and higher in 28%. Seventy-one percent of respondents said their colleagues would agree with the guideline recommendation “very often or always.” Two thirds of physicians indicated that endorsement of the guideline by their colleagues was “moderately or extremely important” to their own treatment decisions.

Rating the Guideline

We asked physicians to rate the helpfulness of the three major guideline components. After their first experience with the guideline, 70% of physicians rated the mortality risk information as helpful, 73% found the triage recommendation for outpatient care helpful, and 73% said the availability of visiting nurse follow-up services was helpful. When we asked physicians which parts of the intervention, if any, they would like to have continued in the future, 91% desired the mortality risk information; 82%, the triage recommendation for outpatient care; and 88%, the visiting nurse follow-up services.

Physicians' perceptions of the guideline's impact on clinical and cost outcomes are shown in Table 2) . Nearly all respondents felt the guideline would decrease hospitalizations and total costs of care. Most physicians also indicated the guideline would improve the quality of care without any increase in serious adverse events, malpractice risk, or patient dissatisfaction. Two thirds of physicians (65%) said their experience with the guideline would make them more likely to treat patients with pneumonia as outpatients in the future, and 66% of them said it made the triage decision easier.

Table 2.

Physician Perceptions of the Impact of the Pneumonia Practice Guideline ^*

Effect of Pneumonia Guideline	Agree, %	No opinion, %	Disagree, %
Fewer hospitalizations	93	6	1
Decreased cost of care	91	7	2
Better quality of care	74	19	7
Easier triage decisions for clinicians	66	18	16
More likely to treat future pneumonia patients as outpatients	65	35	0
Increased patient satisfaction with care	51	28	20
Increased malpractice risk	25	35	40
Increased risk of serious adverse events	18	21	61

^*Survey completed shortly after encounter with eligible pneumonia patient (n= 172).

Postintervention Survey

Rating the Guideline

We surveyed 110 physicians who were still at our institution 1 year after implementation of the guideline (many third-year residents had left). We received 64 (58%) of the postintervention surveys, from 16 (89%) of 18 attending physicians and 48 (52%) of 92 residents. After a year's experience with the protocol, physicians judged the guideline even more favorably than they had after their first interaction with the guideline. Overall, 73% of respondents rated the mortality risk information as helpful, 79% found the triage recommendation for outpatient care helpful, and 94% said the availability of visiting nurse follow-up services was helpful. Ninety-four percent of physicians wanted the mortality risk information to be continued; 97%, the triage recommendation for outpatient care; and 98%, the visiting nurse follow-up services.

Changes in Subsequent Practice, Knowledge, and Atti-tudes

Seventy-one percent of physicians said because of the guideline they were more likely to treat pneumonia patients in the ambulatory setting in the future. In addition, 78% of respondents correctly recalled the 30-day mortality risk for low-risk pneumonia patients stated in the guideline.

In order to further assess the impact of the guideline on pneumonia and guideline attitudes, we analyzed the responses of 58 physicians for whom we had matching preguideline and postguideline survey data. The responses to the two surveys matched for 15 (83%) of 18 attending physicians and 43 (48%) of 92 eligible house officers (Table 1). Attending physicians made up a larger share of respondents with matching data. Otherwise, there were no significant differences among those with and without matching data including no differences in years in practice, gender, pneumonia experience, hours worked, and baseline pneumonia and guideline attitudes (data not shown).

Table 3 shows the effect of the guideline on specific attitudes about pneumonia care. The guideline decreased the belief that, “all patients >65 with pneumonia should be admitted,” from 52% to 14% (p < .0001). Agreement with the statement, “Patients with pneumonia have a>15% mortality rate,” also decreased from 11% to 5% (p = .007). Among the 15 ED attending physicians with matching data (those with the greatest exposure to the guideline), we found an even more dramatic change: at baseline, 61% felt that “all patients >65% with pneumonia should be admitted”; 1 year later, none of them agreed with this statement (p = .0001). The intervention also decreased from 0% to 6%, the proportion of attending physicians who overestimated the pneumonia mortality risk as being greater than 15% (p = .04).

Table 3.

Comparison of Physicians' Attitudes About Pneumonia Care Before and After the Guideline Intervention (n= 58)

	Preguideline		Postguideline
Pneumonia Attitude Questions	Agree, *%	Rating † Mean (SD)	Agree, *%	Rating † Mean (SD)	DifferenceScore ‡	p Value
All pneumonia patients >65 years old should be admitted to the hospital	52	3.0 (1.22)	14	1.96 (1.02)	1.03	.0001
Most patients with pneumonia have a >15% chance of death within 30 days	11	1.96 (0.96)	5	1.57 (0.86)	0.39	.007

^*Percentage who “agreed” or “strongly agreed” with the above statements.

^†Mean rating on a 5-point Likert scale (1 = strongly disagree, 2 = disagree, 3 = no opinion, 4 = agree, and 5 = strongly agree).

^‡Mean difference score is the average difference between an individual physician's postguideline rating and preguideline rating. Significance is based on a paired Student's t test of the mean difference scores.

We had hypothesized that experience with a specific guideline might influence general attitudes about practice guidelines. There was a trend toward greater agreement that “guidelines were good educational tools,” after the intervention (93% postguideline vs 81% preguideline, p = .09; Table 4 However, there were no significant changes in any of the other guideline attitude items or the summary guideline attitude score.

Table 4.

Comparison of Physicians' General Attitudes About Practice Guidelines Before and Afterthe Guideline Intervention (n= 58)

	Preguideline		Postguideline
Attitude Toward Practice Guidelines	Agree, *%	Rating † Mean (SD)	Agree, *%	Rating † Mean (SD)	Difference Score ‡	p Value
A convenient source of advice	79	3.89 (0.85)	88	3.95 (0.88)	0.05	.64
Good educational tools	81	3.87 (0.97)	93	4.10 (0.80)	0.22	.09
Oversimplified or “cookbook” medicine	53	3.19 (1.13)	52	3.14 (1.16)	0.05	.71
A challenge to clinician autonomy	43	2.91 (1.24)	55	3.12 (1.18)	0.21	.24
Too rigid to apply to individual patients	34	2.71 (1.10)	31	2.69 (1.23)	0.017	.92

^*Percentage who “agreed” or “strongly agreed” with the above statements.

^†Mean rating on a 5-point Likert scale (1 = strongly disagree, 2 = disagree, 3 = no opinion, 4 = agree, and 5 = strongly agree).

^‡Mean difference score is the average difference between an individual physician's postguideline rating and preguideline rating. Significance is based on a paired Student's t test of the mean difference scores.

Differences Between Attending Physicians and House Officers

We also compared the responses of attending physicians and house officers because the two groups differed in clinical experience and exposure to the intervention. Attending physicians reported more pneumonia cases than house officers (p = .03), were more likely to be very familiar with the guideline (78% vs 60%, p = .004), and more accurately recalled that low-risk patients had a <2.8% mortality risk (89% vs 79% accurate; p = .08). Compared with attending physicians, house officers rated all three main guideline components (risk information, triage recommendation, and visiting nurse services) even more favorably (p = .02) and were more emphatic about wanting the mortality risk information to be given in the future (p= .007). There was a trend for house officers to prefer that the other guideline components be continued in the future as well (p < .10). They were also more likely than attending physicians to report that the guideline fostered “easier triage decisions” (70% vs 53%, p = .02).

DISCUSSION

There are several ways to measure the impact of practice guidelines. First and foremost, a guideline should be rigorously evaluated to assess whether it produced the desired changes in processes of care. Second, important clinical outcomes should be measured both before and after (or with and without) guidelines to make sure that changing physician behavior does not compromise patient health. We have previously shown that our ED pneumonia guideline changed behavior without compromising patient outcomes.¹³ When consecutive and clinically similar cohorts of patients in the preguideline and postguideline period were compared, the intervention increased the proportion of patients initially treated as outpatients from 42% in the control period to 57% in the guideline period (36% relative increase; p < .01). No patients treated after implementation of the guideline died in the 4-week follow-up period, and rates of symptom resolution, functional recovery, and overall patient satisfaction were not diminished.

The purpose of this study was to extend our evaluation of guidelines to include two other measurements of success that are rarely assessed, one practical and one theoretical. From a pragmatic standpoint, the investigators and hospital managers were interested in physicians' acceptance of and satisfaction with the guideline. Conceptually, we were curious about the intervention's effects on providers' underlying knowledge and attitudes about pneumonia management.

Despite the concern of many critics that guidelines are cookbook medicine and a threat to physician autonomy, we found that physicians had positive opinions about the value and impact of this specific pneumonia admission guideline. More than 70% of physicians rated the guideline as helpful, and nearly all of them wanted it to be continued in the future. Furthermore, ratings after the yearlong intervention were even more favorable than those given after physicians' first experience applying it to an actual patient, which we interpret as another indication of physician satisfaction. Although we were concerned the guideline might be perceived as a threat to clinical autonomy or solely a cost-cutting measure, our data did not bear this out. The majority of physicians felt the guideline made triage decisions easier and resulted in better quality of care.

Little is known about the causal pathways through which guidelines actually change behavior. Ideally, a guideline should alter underlying practice style and, by doing so, produce a sustainable behavior change. However, guidelines may also influence decision making through some combination of a Hawthorne effect, social desirability response, peer pressure, or incentive scheme–altering behavior temporarily or coercively. Our data suggest that the pneumonia guideline changed fundamental beliefs about the necessity of hospitalizing all older patients with pneumonia. This alteration in traditional attitudes about care for the elderly may explain why the guideline achieved the greatest relative reduction in admission rates among older patients.¹³ This finding is particularly significant because the older the patient, the more likely the patient is to be admitted to the hospital, even after controlling for pneumonia severity, comorbidities, sociodemographics, and living arrangements.^15,16 We take as further evidence that changes in attitudes contributed to the change in behavior. Over two thirds of physicians stated that they were more likely to treat patients with pneumonia as outpatients in the future because of their experience with the guideline.

We had hypothesized that a positive experience with one guideline might result in more favorable attitudes about guidelines in general. However, we found only small, insignificant changes in global attitudes. This result may have occurred because general attitudes about guidelines were fairly positive at baseline. Alternatively, the lack of deterioration in general opinions about guidelines over time may itself be notable as there may be secular trends toward more negative attitudes as guidelines are increasingly used as cost-containment tools under managed care.¹² A third explanation is that a physician's response to a specific guideline depends more on the details of its content, context, perceived validity, and implementation strategy than on his or her general attitude of a support or skepticism toward guidelines.

The fact that house officers rated the guideline as more helpful than attending physicians is consistent with the belief that guidelines are “good educational tools” that may be most helpful to users with limited experience or expertise.^3,9,10,17 In a separate multicenter study of pneumonia guidelines, we found that physicians with more pneumonia experience and those with specialty training in pulmonary or infectious diseases were less likely to report being influenced by local and national pneumonia guidelines.¹⁸

There are a few limitations to this study worth acknowledging. The generalizability of our findings may be limited because we studied a modest number of physicians responding to one guideline at one academic institution. However, the response rate was high, pneumonia is a common condition, and our patients were similar to those in a larger multicenter pneumonia study.¹⁴ In addition, the physicians we studied had general attitudes about guidelines similar to those reported among representative samples of U.S. and Canadian physicians.^9,10

It is also worth noting that changing physician knowledge and attitudes might be necessary to produce significant behavior change, but is not necessarily sufficient to do so. There are numerous examples in the literature of performance gaps that do not appear to be due to lack of either physician knowledge or a predisposition to do the right thing. We believe our guideline project succeeded because it incorporated many of the features previously identified as effective. It was developed locally, based on strong evidence and opinion leader input, and implemented in a systematic fashion that included explicit treatment recommendations in real time for specific patients.^1–5,19,20 The ultimate test of the impact of the changes in pneumonia knowledge and attitudes we documented is whether future behavior remains consistent with the guideline, especially as the intensity of implementation is scaled back.

SGIM Website

Please visit the Society of General Internal Medicine on their World-Wide Website. SGIM is located at

http://www.sgim.org