Clinical Decision Support to Improve Antibiotic Prescribing for Acute Respiratory Infections: Results of a Pilot Study

Abstract

Acute Respiratory Infections (ARIs) are the number one reason for antibiotic prescribing in the United States, and much antibiotic prescribing for ARIs is inappropriate. We designed an electronic health record-integrated, documentation-based clinical decision support system for the care of patients with ARIs, the ARI Smart Form. To evaluate the ARI Smart Form and assess the feasibility of performing a larger trial, we conducted a pilot study with 10 clinicians who used the ARI Smart Form with 26 patients. Clinicians prescribed antibiotics to 6 of 6 patients with antibiotic-appropriate diagnoses and to 3 of 20 (15%) patients with antibiotic-inappropriate diagnoses. The average duration of use of the ARI Smart Form was 7.5 (SD±4.5) minutes. Eight of 10 respondents reported that the ARI Smart Form was either time-neutral or timesaving. The ARI Smart Form requires further evaluation but has the potential to improve workflow and reduce inappropriate antibiotic prescribing.

Introduction

Acute Respiratory Infections (ARIs) – including non-specific upper respiratory infections, otitis media, sinusitis, pharyngitis, acute bronchitis, pneumonia, and influenza – are the most common symptomatic reason for seeking ambulatory care in the United States, accounting for approximately 7% of visits.¹ ARIs are also the number one reason for antibiotic prescribing in the United States, accounting for about 50% of antibiotic prescriptions to adults.² Much antibiotic prescribing for ARIs is inappropriate due to prescribing antibiotics for viral conditions or prescribing unnecessarily broad-spectrum antibiotics.³ Inappropriate antibiotic prescribing increases medical costs, increases the prevalence of antibiotic-resistant bacteria, and exposes patients to adverse drug events.

Health information technology, including electronic health records (EHRs) with clinical decision support, has shown the potential for improving the quality of medical care, mainly through the use of prescribing alerts and preventive care reminders.⁴ Improving care for acute problems, like ARIs, may be particularly challenging because of the brevity of ARI visits.⁵ In addition, partially because ARI visits are so brief, research into ARIs is frequently hampered by inadequate and non-standard documentation.

We designed an electronic health record (EHR)-integrated, documentation-based clinical decision support system for the care of patients with ARIs, the ARI Smart Form (Figure 1). The ARI Smart Form has 3 objectives: 1) assist clinicians in reducing inappropriate antibiotic prescribing; 2) improve workflow for clinicians; and 3) improve and standardize documentation. We have previously reported results of usability testing of the ARI Smart Form.⁶ To further evaluate the ARI Smart Form and assess the feasibility of performing a larger randomized controlled trial, we performed pilot testing in actual clinical practice.

Figure 1.

Screen Shot of the ARI Smart Form

Methods

Setting and EHR

Partners HealthCare System is an integrated regional health delivery network in eastern Massachusetts. The main EHR used in Partners HealthCare ambulatory clinics is the Longitudinal Medical Record, or LMR. The LMR is an internally developed, full-featured EHR including typed and dictated notes from primary care and subspecialty clinics; problem lists; medication lists; coded allergies; and laboratory test and radiographic results.

The ARI Smart Form

The ARI Smart Form is launched from the Notes page of the EHR and is designed to be used while interviewing and evaluating patients. The ARI Smart Form includes 6 components: entry of clinical information; patient data display; diagnosis selection; presentation of treatment options with integrated decision support; printing of patient handouts; and access to supporting medical literature. The ARI Smart Form imports patients’ problem lists, allergies, medications, and vital signs into the visit note; speeds workflow using drop-down lists, radio buttons, and check boxes; and provides “one-click” ordering of medicines, patient handouts, and excuse-from-work letters. The ARI Smart Form automatically generates a narrative visit note from all entered information.

The ARI Smart Form provides decision support in several ways. First, clinicians’ selection of a particular ARI diagnosis results in the generation of a diagnosis-appropriate order set. Antibiotic prescribing and antibiotic choices are based on the recommendations of the Centers for Disease Control and Prevention (CDC) and American College of Physicians (ACP).⁷ At a basic level, the ARI Smart Form decision support strives to make the antibiotic treatment match the diagnosis (e.g., not prescribing antibiotics for patients with acute bronchitis). Second, the ARI Smart Form provides diagnostic decision support by calculating the probability of streptococcal pharyngitis based on signs and symptoms and also how rapid streptococcal testing would change the probability of streptococcal pharyngitis (Figure 1).⁸ Third, the ARI Smart Form has medication prescribing alerts to potential medication interactions or patient allergies. Fourth, the ARI Smart Form supports clinicians by providing easy access to diagnosis-appropriate patient handouts. The handouts contain information about the diagnosis and why or why not antibiotics may be indicated.

The ARI Smart Form should standardize documentation for several reasons. To obtain the workflow benefits of using the ARI Smart Form (i.e., facilitated orders and patient instructions), clinicians need to indicate a specific diagnosis as opposed to using non-standard or vague diagnoses like “URI.” In addition, clinicians use the ARI Smart Form to enter standard data elements that are stored and can be used in subsequent analyses.

Pilot Clinicians

We recruited pilot clinicians by emailing 10 clinic directors and asking them to identify 2 volunteers each. Potential pilot clinicians were then contacted via email and invited to participate. Those clinicians who accepted were instructed on how to use the ARI Smart Form through a series of emails detailing the layout, functionality, and technical issues associated with the ARI Smart Form. We encouraged clinicians to practice with the ARI Smart Form by using it with “test patients” (fictitious patients contained in the LMR) and then to start using it for all their ARI visits. On-line support for usability and technical issues was provided for the duration of the pilot via a link in the ARI Smart Form. The pilot period ran from August 29, 2005 to September 31, 2005.

Outcomes

Outcomes of interest for this pilot study included the proportion of all ARI visits for which the ARI Smart Form was used, the proportion of ARI visits at which antibiotics were prescribed, rates of antibiotic prescribing across different diagnoses, and the duration of ARI Smart Form use.

Post-Pilot Survey

At the end of the pilot period, we asked pilot clinicians to complete an electronic survey. The 5-minute survey asked questions about clinicians’ satisfaction with the ARI Smart Form, focusing on areas needing improvement. In particular, the survey asked clinicians if they would recommend the ARI Smart Form to colleagues and if they felt using the ARI Smart Form saved time.

Data Capture, Extraction, and Analysis

We captured data about diagnosis and antibiotic prescribing from the ARI Smart Form. We also recorded time stamps associated with starting and ending the ARI Smart Form session, which allowed us to calculate the duration of use of the ARI Smart Form. The duration of use is only a proxy for the visit duration as clinicians could choose to use the ARI Smart Form throughout a visit or only for a portion of a visit.

After the pilot was complete, we assessed our ability to compare antibiotic prescribing between visits at which the ARI Smart Form was used to visits at which the ARI Smart Form was not used. We extracted 2 groups for comparison. The first comparison group was all contemporaneous ARI visits for which the pilot clinicians did not use the ARI Smart Form. The second comparison group was all ARI visits from the previous winter season (from October 1, 2004 to May 31, 2005) made to the pilot clinicians.

We identified comparison visits using administrative data coded as International Classification of Diseases, Ninth Edition, Clinical Modification (ICD-9-CM). We identified visits with diagnoses of non-specific upper respiratory infections (ICD-9-CM 460, 464, and 465), otitis media (ICD-9-CM 381 and 382), sinusitis (ICD-9-CM 461 and 473), pharyngitis (ICD-9-CM 034.0, 462, and 463), acute bronchitis (ICD-9-CM 466 and 490), and pneumonia (ICD-9-CM 481–486). We considered otitis media, sinusitis, streptococcal pharyngitis, and pneumonia to be antibiotic-appropriate diagnoses. We considered non-specific upper respiratory tract infections, non-streptococcal pharyngitis, acute bronchitis, viral syndrome, and other diagnoses to be non-antibiotic-appropriate diagnoses.

We report results as simple proportions and means with standard deviations (SD). We did not perform statistical testing because of the small sample size of this pilot study.

Results

Clinician Participation and Characteristics

We identified and invited 17 clinicians to use the ARI Smart Form. Sixteen of 17 used the ARI Smart Form on either test or real patients. A total of 58 notes were generated via the ARI Smart Form for test and real patients for an average of 3.6 notes per clinician. During the pilot period, 10 clinicians used the ARI Smart Form 26 times with real patients, including 15 of 54 actual patient visits (28%) with an ICD-9-CM code of an ARI and an additional 11 visits that did not have an ARI ICD-9-CM.

This group of 10 Partners-affiliated physicians represented 9 different practices in the Partners network. Although nurse practitioners were among those invited to participate in the pilot, all 10 participants who used the ARI Smart Form with real patients were physicians. These clinicians included 5 women, had a mean age of 42 (SD±6.7) years old, and, on average, graduated from medical school 15 years previously. Nine of the pilot clinicians had primary care practices and 1 saw only urgent care patients.

Patient Characteristics

The mean age of the 26 patients for whom the ARI Smart Form was used was 44 (SD±15) years old and included 15 (60%) women. Of these patients, 17 (65%) were white, 2 (8%) were Latino, and 7 (27%) had unknown race and ethnicity. Twenty-four patients (92%) spoke English as their primary language.

Antibiotic Prescribing

Overall, during the pilot period, clinicians prescribed antibiotics to 35% (9 of 26) of patients when using the ARI Smart Form and 38% (15 of 39) of patients when not using the ARI Smart Form for ARI visits (Table 1). During the previous influenza season, these same clinicians prescribed antibiotics in 30% of ARI visits. For antibiotic-appropriate diagnoses, clinicians prescribed antibiotics in 6 of 6 visits (100%) when using the ARI Smart Form, 9 of 10 visits (90%) when not using the ARI Smart Form and in 154 of 367 visits (42%) during the previous cold and influenza season. For antibiotic-inappropriate diagnoses, clinicians prescribed antibiotics in 3 of 20 visits (15%) when using the ARI Smart Form, 6 of 29 visits (21%) when not using the ARI Smart Form, and 269 of 1027 visits (26%) during the previous cold and influenza season.

Table 1.

Antibiotic Prescribing by Diagnosis

	Pilot Period				Previous Season
	Smart Form		Non-Smart Form
	Visits, N	Antibiotic, N (%)	Visits, N	Antibiotic, N (%)	Visits, N	Antibiotic, N (%)
Antibiotic Appropriate Diagnoses
Otitis media	0	NA	1	1 (100)	54	29 (54)
Sinusitis	3	3 (100)	7	7 (100)	188	96 (51)
Streptococcal pharyngitis	3	3 (100)	1	1 (100)	4	4 (100)
Pneumonia	0	NA	1	0 (0)	121	25 (21)
Sub-Total	6	6 (100)	10	9 (90)	367	154 (42)
Non-Antibiotic Appropriate Diagnoses
Non-specific upper respiratory infection	8	2 (25)	15	4 (27)	578	130 (22)
Non-streptococcal pharyngitis	2	0 (0)	2	1(50)	260	72 (28)
Acute bronchitis	7	1 (14)	1	1(100)	167	65 (39)
Viral syndrome	2	0 (0)	4	0 (0)	22	2 (9)
Other	1	0 (0)	7	0 (0)	NA	NA
Sub-Total	20	3 (15)	29	6 (21)	1027	269 (26)
Total	26	9 (35)	39	15 (38)	1394	423 (30)

^*NA is not applicable

Visit Duration

The mean duration of ARI Smart Form use (calculated from the time the Smart Form was opened to the time the Smart Form was completed) was 7.5 minutes (SD±4.5), ranging from 2.0 minutes to 18.7 minutes (Table 2). The duration of use appeared to generally decrease with the number of uses by clinicians though there was notable variability within and between clinicians and 3 clinicians only used the ARI Smart Form one time.

Table 2.

ARI Smart Form Use Duration by Clinician and Number of Uses

Clinician	Uses of the ARI Smart Form
	1st	2nd	3rd	4th	5th	6th
	Time (Minutes)
1	10.2
2	14.8	7.0	4.9	8.7	5.0
3	4.1	10.5
4	6.3	7.9	8.5	14.5
5	4.1	3.6
6	8.3	7.6
7	13.2
8	7.0
9	8.6	18.7
10	3.5	3.8	4.4	3.3	2.0	3.8
Mean	8.0	8.4	5.9	8.8	3.5	3.8

Survey Results

Ten pilot clinicians responded to the post-pilot survey. Three clinicians felt the ARI Smart Form was marginally timesaving, 5 felt it was time-neutral, 1 felt it marginally increased work, and 1 felt it significantly increased work. Six clinicians would recommend that other clinicians use the ARI Smart Form unmodified and 3 would recommend it with some minor modification, such as increasing flexibility with more “freelance choices” and the feeling that the final note did not “flow naturally.”

Discussion

In this pilot, we demonstrated the feasibility of conducting a larger study to evaluate the ARI Smart Form. We found that the ARI Smart Form has the potential to decrease inappropriate antibiotic prescribing for ARIs and improve workflow. The antibiotic prescribing rate for all ARIs did not change, but clinicians appeared to increase antibiotic prescribing for antibiotic-appropriate diagnoses and decrease antibiotic prescribing for non-antibiotic-appropriate diagnoses. This is in line with a major objective of the ARI Smart Form decision support: to get the antibiotic treatment of ARIs to match the diagnosis. Achieving such a goal could have a major impact on reducing overall antibiotic prescribing in the United States.²

A major barrier to the adoption of EHRs is the perception that they interfere with workflow. Encouragingly, pilot clinicians generally felt that the ARI Smart Form was time-neutral or timesaving. This perception is in agreement with the measured results of local time-motion studies.⁹ It is especially promising that, with minor modifications, 9 of 10 pilot users who used the ARI Smart Form with actual patients would recommend the ARI Smart Form to their colleagues.

The ARI Smart Form should provide additional benefits by standardizing documentation for ARIs. Some structured data elements are required for ARI Smart Form decision support at the time of use, like the signs and symptoms for calculating the probability of streptococcal pharyngitis. In addition, structured information can be used for quality reporting and quality improvement initiatives. Indeed, we are implementing an ARI “Quality Dashboard,” with which clinicians review their antibiotic prescribing rates for ARIs compared to colleagues and national averages.

Limitations

This pilot study has several limitations that should be considered. First, and most obviously, this study was small with few participating clinicians using the ARI Smart Form with actual patients. This limited our ability to perform more meaningful comparisons between groups and statistical testing. Second, apparent improvements in appropriate antibiotic prescribing may simply reflect learning by clinicians to better match diagnosis codes with prescribed treatments. Third, not all of the invited clinicians used the ARI Smart From. Clinicians who are more inclined towards applications like the ARI Smart Form might assess it more favorably, leading to response bias in our satisfaction measures. Fourth, though we have previously found that administrative diagnoses have good positive predictive value for identifying ARI visits,¹⁰ in this pilot, there was not a 1:1 relationship between the administrative diagnosis and the diagnosis from the ARI Smart Form. Fifth, participating clinicians did not use the ARI Smart Form for all of their ARI patients. There could be systematic differences between patients and visits at which clinicians do and do not choose to use the ARI Smart Form.

Future Directions

The ARI Smart Form requires further evaluation with a larger sample of clinicians and patients in a randomized controlled trial. Because our experience in the present pilot indicates that clinicians will probably use the ARI Smart Form for a minority of ARI visits, such a trial needs to have a large sample size. An adequately powered trial could include a broader range of outcomes, such as re-visit rates, antibiotic costs, use of broader-spectrum antibiotics, and quality of documentation. We also hope to examine if there is an inverse association between the number of problems addressed in a visit and the use of the ARI Smart Form. In assessing the appropriateness of antibiotic prescribing in a larger study, we need to be aware of the potential for increasing the use of antibiotic-appropriate diagnoses and inadvertently increasing antibiotic prescribing for ARIs. For this reason, the primary outcome should be the overall antibiotic prescribing rate for all ARIs combined. Within this primary outcome, one can detect “diagnosis-shifting” from non-antibiotic-appropriate diagnoses to antibiotic-appropriate diagnoses.

Further changes to the application are also planned, including better integration of the ARI Smart Form into a visit in which multiple medical problems are addressed, increasing the flexibility of the form, and improving the readability of the generated note.

Conclusion

In this pilot study, we found that the ARI Smart Form did not appear to change the overall antibiotic prescribing rate. We did find the ARI Smart Form has the potential to decrease the antibiotic prescribing rate for non-antibiotic-appropriate diagnoses. We also found that pilot clinicians felt that the ARI Smart Form was time-neutral or timesaving and would generally recommend it to colleagues. Decision support applications for acute problems must provide clinicians with self-evident benefits at the time of the visit (e.g., saving time, improving patient education) or they will go unused. The ARI Smart Form requires further evaluation, but has the potential to improve workflow, reduce inappropriate antibiotic prescribing, and standardize documentation.