Transforming User Needs into Functional Requirements for an Antibiotic Clinical Decision Support System

Summary

Background

Many informatics studies use content analysis to generate functional requirements for system development. Explication of this translational process from qualitative data to functional requirements can strengthen the understanding and scientific rigor when applying content analysis in informatics studies.

Objective

To describe a user-centered approach transforming emergent themes derived from focus group data into functional requirements for informatics solutions and to illustrate these methods to the development of an antibiotic clinical decision support system (CDS).

Methods

The approach consisted of five steps: 1) identify unmet therapeutic planning information needs via Focus Group Study-I, 2) develop a coding framework of therapeutic planning themes to refine the domain scope to antibiotic therapeutic planning, 3) identify functional requirements of an antibiotic CDS system via Focus Group Study-II, 4) discover informatics solutions and functional requirements from coded data, and 5) determine the types of information needed to support the antibiotic CDS system and link with the identified informatics solutions and functional requirements.

Results

The coding framework for Focus Group Study-I revealed unmet therapeutic planning needs. Twelve subthemes emerged and were clustered into four themes; analysis indicated a need for an antibiotic CDS intervention. Focus Group Study-II included five types of information needs. Comments from the Barrier/Challenge to information access and Function/Feature themes produced three informatics solutions and 13 functional requirements of an antibiotic CDS system. Comments from the Patient, Institution, and Domain themes generated required data elements for each informatics solution.

Conclusion

This study presents one example explicating content analysis of focus group data and the analysis process to functional requirements from narrative data. Illustration of this 5-step method was used to develop an antibiotic CDS system, resolving unmet antibiotic prescribing needs. As a reusable approach, these techniques can be refined and applied to resolve unmet information needs with informatics interventions in additional domains.

1. Introduction

Implementing a user-centered approach [1-4] to evaluate health information technology (health IT) applications, e.g., clinical information system (CIS), computerized prescribing order entry (CPOE) system, clinical decision support (CDS) system, within the systems development life cycle (SDLC) has yielded a greater understanding of the interaction between systems and users [5, 6]. As an informatics intervention, CDS has enabled clinicians to better address arising information needs with evidence-based guidelines to deliver the best available care; however, evidence demonstrating efficacy on clinical and economic outcomes remains sparse despite improvements in CDS functionality [7]. One possible method to increase realization of some health IT benefits is to design applications that examine and incorporate users as the focal point of the system development process. There is a dearth of informatics articles expounding the process of transforming data gathered from qualitative sources (e.g., evaluative workshops, focus groups, interviews, and observations) with users into functional requirements for CIS and CDS application design [3,4,8,9]. Explicating this process can reinforce the understanding and scientific rigor when applying qualitative methods to guide system development.

Within the infectious diseases (ID) domain, the complexity of clinical tasks necessitates a comprehensive awareness of the relationship between user information needs and functional requirements for CDS systems. The singular task of judicious antibiotic prescribing involves major decision points, including:

1. deciding to use antibiotics based on the suspicion of an infection and ID knowledge,

2. selecting the appropriate antibiotic based on the likely pathogen, antibiogram, clinical significance, side effects, formulary, cost, patient preferences, and clinical condition, and

3. selecting the appropriate dosage, route, interval, and duration of therapy [10,11].

The expectation is that CDS systems integrated into antibiotic stewardship programs will lead to sustained impacts on clinical outcomes and process measures by adjusting prescribing behavior [12]; however, there is still limited penetration of these systems in clinical practice demonstrating efficacy [13-16].

2. Objectives

To describe a user-centered approach transforming emergent concepts and themes derived from focus group data into functional requirements for informatics solutions capable of addressing unmet user information needs. To apply these qualitative methods to the development of an antibiotic CDS system, producing a visual knowledge structure to inform future system development and evaluation.

3. Methods

3.1. Sample and Setting

The study setting was Columbia University Medical Center/NewYork-Presbyterian Hospital (CUMC/NYP) for two focus group studies, where a commercial CPOE system providing basic antibiotic decision support functionality was implemented. Several CUMC/NYP clinicians used email and person-to-person communications to recruit participants across the various clinical groups. The author, who had previous experience leading focus groups, conducted the sessions. A colleague also with expertise in the focus group methodology took extensive notes. Participants received a $50 or $75 gift card at the conclusion of the session. Food and beverages were also provided.

3.2. Procedures

The user-centered approach consisted of five steps. First, unmet therapeutic planning information needs were identified via a series of focus group studies. A focus group guide consisting of open-ended questions [17] was used to elicit participant responses (► [Appendix A and ► [Appendix B). Each focus group session lasted for one hour. Participant verbalizations were captured through detailed notes and audio recordings. Focus Group Study-I was conducted to gain an understanding of the therapeutic planning process and to identify clinical tasks where informatics solutions could address unmet user information needs. Discussion centered on the components of therapeutic planning and situations or conditions that potentially affect the planning process. The study included three focus group sessions, one with each clinical user group. The inclusion criteria included:

1. CUMC/NYP Resident/Fellow Physician, Attending Physician, or Nurse Practitioner, and

2. experience in creating therapeutic plans.

The three session recordings were transcribed verbatim. Second, the coding framework was developed using the inductive content analysis methodology [18,19]. Transcripts were analyzed to discover concepts and themes related to unmet therapeutic planning information needs using a six-phase approach to thematic analysis [20]. After familiarizing oneself with the data, initial codes were generated from each participant statement that described information need concepts with some data extracts having multiple codes. Next, collation of related codes and the relationship between codes identified broad themes and subthemes pertaining to therapeutic planning tasks. The coded data extracts were reviewed for coherency within each theme and within the context of the data set to ensure accurate representation of the data. A coherent, consistently generated, and distinct coding framework of codes and themes required some recoding to improve clarity and reduce and the volume of categories. Lastly, the themes and subthemes were given a more descriptive name and ongoing refinement occurred as needed. Peer debriefing and audit trails were implemented to establish qualitative rigor of the data collection and analysis phases [21-24]. Peer debriefing with research team members discussing methodological steps helped minimize reactivity and bias. An audit trail consisting of summaries, transcripts, coding schemes, and process notes provided a clear description of how emergent themes were identified from the narrative data; another team member reviewed these artifacts on a continuous basis, verifying the clarity and consistency of the analysis. In both cases, consensus was reached through discussion and modifications were incorporated into the codebook. The resulting emergent themes narrowed the scope of the application domain and informed the research design of the second focus group study to gather functional requirements for the informatics solution.

Third, focus groups were convened (Focus Group Study-II) to elicit functional requirements of an antibiotic CDS to address unmet antibiotic therapeutic planning tasks. The study included two focus group sessions, one with non-ID clinicians and the other with ID clinicians. Inclusion criteria for the non-ID clinician focus group included:

1. CUMC/NYP Resident/Fellow Physician, Attending Physician, or Nurse Practitioner,

2. non-ID specialty and appointment, and

3. prescribing clinician who enters orders in the CPOE system.

Inclusion criteria for the ID expert focus group included:

1. CUMC/NYP Resident Physician, Attending Physician, Nurse, Nurse Practitioner, or Clinical Pharmacist, and

2. ID specialty and appointment.

The audio recordings were transcribed verbatim by a transcription service with transcripts verified by the study author. A portion of the Information Needs Event Taxonomy [8] was selected as an appropriate coding framework to ascertain antibiotic therapeutic planning information needs and identify functional requirements for an antibiotic CDS system. Themes 1–3 were based on the Information Needs Event Taxonomy [8], and the complete coding framework included a total of five themes. The classified information needs were: (1) Patient, (2) Institution, or (3) Domain; themes 4 and 5 were based on unmet information needs that emerged from Focus Group Study-I pertaining to: (4) Barrier/Challenge to information access or (5) Function/Feature.

During the first step of analysis, each statement was coded for one theme at a time, beginning with the first category of Currie’s taxonomy [8], Patient information need; this was repeated for the remaining themes. At the conclusion of this step, the individual pieces of data within each statement had multiple codes. Similar to Braun’s method [20], there was a process of recoding, ensuring that all data extracts were identified and appropriately coded. Since the thematic categories were defined a priori using a deductive content analysis approach [18] with NVivo© (QSR International Pty, Doncaster, Victoria), several phases of Braun’s method to construct the coding framework were not applicable. As detailed above, peer debriefing and audit trails were employed to delineate methodological steps and decision points for managing and analyzing data.

Next, informatics solutions for implementation and functional requirements of an antibiotic CDS system were identified from the Barrier/Challenge to information access and Function/Feature coded data extracts. Last, the types of information needed from the Patient, Institution, and Domain coded data extracts to support the antibiotic CDS system were linked with the identified informatics solutions. Another evaluator with ID expertise validated these artifacts and consensus was achieved iteratively through discussions and revisions.

4. Results

► Figure 1 presents an illustration of the approach to transform emergent concepts and themes derived from focus group data into functional requirements for an antibiotic CDS system. Twelve clinicians participated in Focus Group Study-I: 4 Resident Physicians, 2 Attending Physicians, and 6 Nurse Practitioners. Ten were female and two were male. All participants had experience creating therapeutic plans in the inpatient setting and represented a range of clinical specialties including internal medicine (2), neurosurgery (2), psychiatry (2), critical care (1), endocrinology (1), infectious diseases (1), nephrology (1), pain management (1), and women’s health (1). Twelve clinicians completed Focus Group Study-II: 6 Resident/Fellow Physicians, 3 Nurse Practitioners, and 3 Clinical Pharmacists. Seven were female and five were male. Three (1 Fellow Physician and 2 Clinical Pharmacists) had an ID appointment at CUMC/NYP and the remaining respondents had clinical appointments other than ID.

Fig. 1.

Overview of the user-centered approach transforming focus group data into functional requirements for an antibiotic CDS

The coding framework for Focus Group Study-I identified unmet informational needs during the therapeutic planning process. Comments revealed the need for an antibiotic CDS intervention, narrowing the scope of the application domain to antibiotic therapeutic planning for Focus Group Study-II. ► Figure 2 displays an annotated participant comment expressing the need for an antibiotic history timeline to track the history of administered antibiotics based on current Barriers/ Challenges to information access. The statement also included Patient information needs required to complete the clinical task of reconciling past antibiotic orders.

Fig. 2.

Example of a participant comment coded to identify Patient Data and Barrier/ Challenge to information access necessary when reconciling antibiotic medication history

Three hundred and eighty-five statements were recorded from Focus Group Study-I, generating 426 information needs. Ninety-seven concepts were identified and collated into twelve subthemes and four themes related to therapeutic planning information needs: Communicate, Gather, Document, and Plan (► Figure 3). Examples of participant comments are presented in ► Table 1. The type of information needed requiring a discussion with other providers was expressed in the following Communicate theme statement, “Page someone [ID] and get a drug recommendation, not a consult; just, ‘I was thinking this and wanted backup to say yes this is appropriate or not’.” Nurse Practitioner, Subtheme: communication type. The following Gather theme statement describes information needed to be collected for an assessment, “The name, the objective things that you know about it, the subjective things that you know about it, and then the appraisal from those things and then a plan.” Attending Physician, Subtheme: information type. The process of identifying information needs when writing notes was captured in the following Document theme statement, “Usually when writing my note I think of more things. Then you are at a computer anyway, so you order more lab tests for the following morning.” Resident Physician, Subtheme: documentation method. The types of information needed when formulating a therapeutic plan was articulated in the following statement, “Routine medications I know most of the side effects, but a lot of medications that you only prescribe once in awhile, especially antibiotics you may not know. So I always check myself. It takes like two seconds.” Attending Physician, subtheme: information needs.

Fig. 3.

Focus Group Study-I, Therapeutic planning concept codes and themes

Table 1.

Focus Group Study-I, Therapeutic Planning Focus Group Information Needs

Theme	Subtheme	Example
Communicate	Communication Type	“Call that process ‘curbside consult’. Page someone [ID] and get a drug recommendation, not a consult; just, ‘I was thinking this and wanted backup to say yes this is appropriate or not’.”
	Data Accessibility	“Especially with ID because every six months they change their drug of choice/ protocol.”
	Collection Challenges	“And how do you get all that information? You don’t do it in 5 minutes. You may not do it in 5 hours. But, you feed data in anyway you can.”
Gather	Information Type	“The name, the objective things that you know about it, the subjective things that you know about it, and then the appraisal from those things and then a plan.”
	Information Source	“WebCIS resources: drug, Infectious Diseases sources.”
Document	Documentation Method	“Usually when writing my note I think of more things. Then you are at a computer anyway, so you order more lab tests for the following morning.”
	Documentation Medium	“But as long as he can learn to use the machine, which your generation all can and a lot of my generation doesn’t use the machine very well. And so it’s really like pulling nails to get them to put their records in and their orders in on the computer.”
Plan	Define Plan Goals	“Two levels at night when admitting and planning: 1) what does patient need acutely to be stabilized (antibiotics, urgent neuro consult); and 2) what to do in order to get patient ‘tucked’- tucked in and not going to die on you…”
	Medication Plan	“So kind of like taking into account all of their problems. For each problem, thinking is there a certain medication they should have? If they come in with a fever, what antibiotic should they have?”
	Therapeutic Plan Scope	“… an internist is really on the other side of the coin [and] really has to look at it as a function of everything that the patient has lived through, gone through, and their genetic predispositions...”
	Information Needs	“Routine medications I know most of the side effects, but a lot of medications that you only prescribe once in awhile, especially antibiotics you may not know. So I always check myself. It takes like two seconds.”
	Errors	“People make documentation errors; not always accurate- left side is right side and [you] have to check. Electronic documentation doesn’t make [it] easier.”

A total of 239 verbalizations were captured from Focus Group Study-II, producing 401 information needs. Thirteen functional requirements emerged from data analysis for an antibiotic CDS system and were grouped into three informatics solutions: infobutton, alert, and dashboard (► Table 2). Alert classification was based on a clinical alerts taxonomy by Chused et al. [25]. ► Table 3 displays examples of antibiotic therapeutic planning information needs expressed by participants. When formulating the antibiotic therapeutic plan, a Non-ID Resident Physician expressed needing the following Patient information, “Look at their microbiology history and see if they have reoccurring infections with particular organisms, which [ones], what is the susceptibility of the organisms that they tend to be infected with.” When completing this task, another Non-ID Resident Physician stated a need for Institution information, “You could go on the infectious disease website and look up what drugs are done early covering which organisms and just their patterns to what covers what.” An ID Fellow recalled identifying needed Domain information in the following statement, “Oh, just in terms of clinical, good practice guidelines over what’s more efficacious, whether it’s journal articles or [the] Stanford guide.”

Table 2.

Focus Group Study-II, Functional Requirements for an Antibiotic CDS System

Informatics Solution	Functional Requirement
1. Infobutton	1. Antibiotic Selection Tool
2. Alert	Critique Alert	2. Antibiotic-Microorganism Mismatch
		3. Antibiotic Dosing (Under and Over)
		4. De-escalation/Streamlining
		5. Antibiotic Order Modification
	Informational Alert	6. New/Update Status
		7. Order Administration
		8. Check for Past Culture History
	Suggestion Alert	9. Recalculate glomerular filtration rate (GFR)
		10. New/Update Status with Recommended Action
		11. Review One-Time Antibiotic Order
		12. Antibiotic Order, No Culture
3. Dashboard		13. Antibiotic History Timeline

Table 3.

Focus Group Study-II Antibiotic Therapeutic Planning Information Needs

Theme	Example
Patient	“Look at their microbiology history and see if they have reoccurring infections with particular organisms, which [ones], what is the susceptibility of the organisms that they tend to be infected with.”
	“Someone will say in their notes, you know, admitted with pneumonia, started on coverage for community-acquired pneumonia or whatever and they’ll be pretty clear.”
Institution	“I know in our medical ICU we keep an updated board for ourselves and usually I think they are putting that on the MICU boards now, what the antibiotic is treating. But that is something we do outside of WebCIS and outside of Eclipsys for ourselves because we know it is a problem.”
	“You could go on the infectious disease website and look up what drugs are done early covering which organisms and just their patterns to what covers what.”
Domain	“Drugs like Linezolid, for instance, are well known to cause thrombocytopenia, but not everybody remembers, and it is especially important if someone is thrombocytopenic.”
	“Oh, just in terms of clinical, good practice guidelines over what’s more efficacious, whether it’s journal articles or [the] Stanford guide.”
Barrier/ Challenge to information access	“And then on the other end of the spectrum, ordering antibiotics and not stopping them when you’re supposed to. You know, after three days, five days, seven days, and then just leaving them on continuously.”
	“Sometimes people don’t think to go back and there’s almost like a patient safety stage there, where people kind of say ‘I’m going to choose X antibiotic.’ But a year ago, it was resistant, but they didn’t pay attention to that.”
Feature/ Function	“I guess there are different levels that you could think about it, whether you’re just going to do this mismatch program for cultures that have been identified and grown, and that they’re susceptibility testing for. Or, you’re going to ask whoever is inputting data to say what they’re empirically treating. And then if it’s a UTI, then vanco [mycin] is not a match.”

As depicted in ► Figure 4, Barrier/Challenge to information access and Feature/Function coded data extracts generated the informatics solutions and necessary functional requirements. An example of a Barrier/Challenge to information access expressed by an ID Pharmacist was “Sometimes people don’t think to go back and there’s almost like a patient safety stage there, where people kind of say ‘I’m going to choose x antibiotic.’ But a year ago, it was resistant, but they didn’t pay attention to that.” Informatics solution: alert, Functional requirement: check for past culture history informational alert. An example of a Feature/Function expressed by an ID Pharmacist was “Absolutely, like kind of an antibiotic history. And if you can make it graphic, that’s even better because it’s easier to follow.” Informatics solution: dashboard, Functional requirement: antibiotic history timeline. ► Figure 5 displays the relationships between the informatics solutions generated from the Barrier/Challenge to information access and Feature/Function coded data extracts and the necessary Patient, Institution, and Domain information needs identified from the focus group data to support antibiotic prescribing tasks. These functional requirements guided the future development and evaluation of an antimicrobial-microorganism ontology-based CDS tool [26].

Fig. 4.

Focus Group Study-II, Translation of Barrier/Challenge to information access and Function/Feature needs to functional requirements for an antibiotic CDS system

Fig. 5.

Focus Group Study-II, Types of information needs to support informatics solutions for an antibiotic CDS system

5. Discussion

This study reports on a reusable, user-centered approach for transforming emergent concepts and themes derived from focus group data into a knowledge structure representing the relationships between user information needs, functional requirements, and informatics solutions for an antibiotic CDS system. Focus Group Study-I yielded an understanding of the therapeutic planning process and identified unmet information needs regarding the antibiotic therapeutic planning process. Focus Group Study-II enumerated on user needs for antibiotic therapeutic planning and emergent themes influenced the functional requirements for informatics solutions, consequently delineating the scope of the resulting antimicrobial-microorganism ontology-based CDS intervention. The two focus groups were necessary to narrow the scope in this instance, but not essential to the general methodology.

The author believes that this is the first study to expound the data analysis procedure of developing a coding framework from a qualitative data set using content analysis within the informatics domain. Content analysis is a frequently utilized approach; however, there is a paucity of biomedical informatics examples using content analysis to develop functional requirements. As discussed in the literature, researchers can have unresolved methodological uncertainties because of the poorly delineated and defined descriptions [18-20]. Explication can strengthen the understanding and scientific rigor when applying content analysis in informatics studies. Previous studies applied inductive or deductive analyses to ascertain information and needs, but did not explicate their process of developing functional requirements for informatics interventions from narrative text and themes [1,3,4,8,27,28]. A similar pattern was noted within a broader informatics context where studies provided few details of how themes from qualitative data emerged using grounded theory [1,4,29,30]. One study expounded on the coding process to develop organizational factors pertaining to the use of information technology for clinical practice guidelines [31], but this was not elucidated in many other good studies [3,23,32,33]. Each of the studies referenced here contributed knowledge regarding functional requirements and for many, the authors were able to later publish on implemented systems, informed by this intermediary step. This study presents a complementary approach when conducting the system analysis phase, given the significance and frequency of qualitative methods used to identify emergent themes for functional requirements.

The findings in this study illustrate useful lessons for conducting the SDLC analysis phase. Generating a knowledge structure that visually represented how information solutions and necessary functional requirements from user information needs yielded a product that clearly aligned informatics solutions for antibiotic prescribing tasks with users’ expressed needs and desired system functionality. As part of the validation process with the ID expert, this artifact iteratively evolved, resulting in a document that described information about the antibiotic prescribing tasks and system behavior for proposed system functionality in an unambiguous and consistent manner. Further, our methods of identifying unmet information needs and currently available informatics solutions maintained user involvement throughout the functional requirements definition process. Mapping information needs from the focus group studies to functional requirements of informatics solutions for antibiotic therapeutic planning also provided an overall view of unmet antibiotic prescribing information needs and challenges with the current CPOE functionality, thus enabling informed decisions to be made regarding future system design and guiding the scope of the antibiotic CDS system. Future work will examine the impact of this user-centered approach on the system design process, provider acceptance, and patient outcomes.

There are limitations of the generalizability of this study. Both focus group studies occurred at one large academic medical center where Resident and Fellow Physicians are active contributors to the therapeutic planning process. Participants also used a combination of a commercial CPOE system and a locally-developed CIS to perform therapeutic planning tasks; clinician experiences could vary with different systems. Importantly, since the study did not provide direct observation of clinicians when information needs transpired, therefore eliminating the option of a grounded theory analysis, information needs were self-reported and subject to recall biases. Some themes achieved data saturation, however, this was a relatively small sample size. Lastly, although this iterative design approach was peer validated with experts in system development, a dedicated system analyst was not employed, which would strengthen the process of translating user needs into functional requirements.

6. Conclusion

This study presented one example explicating content analysis of focus group data and the analysis process to produce informatics solutions and functional requirements from narrative data. Illustration of this 5-step method was used within the context of antibiotic therapeutic planning to guide the development an antibiotic CDS system to resolve unmet information needs. Steps 1 through 3 provided insight into the complexity of therapeutic planning and narrowed the scope of the intervention to address unmet information needs for antibiotic prescribing tasks. Steps 4 and 5 utilized participant comments regarding barriers or challenges when synthesizing data and information and desired functionality of antibiotic therapeutic planning tools to inform the identification of informatics solutions, functional requirements, and necessary data to support each solution. As a reusable approach, these techniques can be refined and applied to resolve unmet information needs with informatics interventions in additional clinical domains. A subset of the identified functional requirements of an antibiotic CDS informed the development and evaluation of an antimicrobial-microorganism ontology-based CDS to guide antibiotic prescribing.

Clinical Reference Statement

Findings from this study demonstrate the usefulness in explicating the critical step of translating concepts and themes obtained through qualitative analysis into functional requirements for informatics solutions. Our results emphasize the development of a novel approach to assist researchers in creating knowledge artifacts to represent the linkages and relationships between user information needs, functional requirements, and informatics solutions, while also maintaining user involvement in the design to guide future system development.

Conflict of Interest

The author declares that there are no conflicts of interest in the research.

Protection of Human and Animal Subjects

The Columbia University Institutional Review Board approved all research procedures.

Appendix A Focus Group Study-I Guide

Number	Category	Question
1	Introduction	Based on your experience and knowledge, please tell me a general definition and components of therapeutic planning?
	Probe	Is your definition based on a therapeutic planning framework or by personal experience? Is there a difference in the way therapeutic planning is taught compared to the way it is actually performed in practice?
2	Transition	How do you determine the goals of therapeutic planning for each patient?
	Probe	Prevent a disease... slow the progression of a disease… cure a disease… diagnose a disease… reduce symptoms?
3	Key	Think back to the last time you created a therapeutic plan for a patient. Would you describe the steps you took to document the therapeutic plan?
4	Key	Think back to the last time you needed additional information to complete the therapeutic plan. What was the type of information and where did you go for the information?
	Probe	Do you rely on co-workers as a knowledge source? Do you access the literature or the Internet for reference?
5	Key	In your opinion, what is the ‘right’ way to document the therapeutic plan?
	Probe	Systems-based or problem-based approach? What are the strengths and weaknesses of the approach?
6	Ending	Do you have any final suggestions for us to consider?

Appendix B Focus Group Study-II Guide

Number	Category	Question
1	Introduction	Based on your experience and knowledge, please tell me how you would define an antibiotic-microorganism mismatch or bug-drug mismatch?
	Probe	How do you determine if there is a mismatch?
2	Transition	Does antimicrobial resistance factor into your decision when you are creating an antibiotic plan or assisting with a plan?
	Probe	Are you concerned about resistance? Are you more likely to order broad-spectrum antibiotics?
3	Key	Think back to the last time you ordered an antibiotic. What type of data did you need to complete the antibiotic plan (other than microbiology results) and where did you go for the information?
	Probe	Medication information from a pharmacy knowledge base? Literature?
4	Key	In your opinion, what type of information should an antibiotic-microorganism mismatch alert provide? What should it check?
	Probe	Updated culture results? Suggest alternative antibiotic therapy?
5	Key	In your opinion, what type of features would you like to see in an antibiotic-microorganism mismatch alert?
	Probe	Timeline of ordered antibiotics? Possible antibiotic resistance risk factors? Relevant literature about an organism or antibiotic?
6	Key	Think back to the last time you ordered an antibiotic for a patient. Why did you order the antibiotic and would you describe the steps you took to order the drug?
	Probe	Was the decision based on culture results? Once you made a decision to order the drug, what did you do next?
7	Key	Do you have any final suggestions for us to consider?
8	Key	What are the problems with the current method of prescribing antibiotics?