Table 4:

Components of workflow integration related to clinician outcomes

#	Components of workflow integration (# of interviewees)	Description (− indicates a barrier to workflow integration; + indicates a facilitator to workflow integration)	Clinician outcomes
			Positive	Negative
21	Hard to find PE Dx and remember to use (4 physicians – residents)	(−) PE Dx is not integrated in the workflow so the physician forgets to use the tool and/or has a hard time remembering to use the tool. It is challenging to find the CDS in the EHR the first time using it which is a hurdle to remembering to use it. PE Dx would be more integrated in the workflow if it was not in the ED navigator because that part of the EHR is rarely visited, and if it is used, it is not until the end of the ED visit when a physician is ready to write the notes.		• Lack of adoption/use
22	Physician was not initially aware of PE Dx (3 physicians – residents)	(−) Some physicians were not initially aware that PE Dx existed because it is in the ED navigator, which they do not often use. PE Dx is not integrated in the workflow because the physician does not think to use the PE Dx in their workflow. The physician goes to MDCalc or straight into the notes and documents their medical decision making without thinking to use PE Dx. When attending physicians are at UW, they do not think to use PE Dx because the residents are the ones placing most orders and writing the notes.		Lack of use
23	Impact on efficiency, time to complete tasks and workload (7 physicians - 3 attendings, 4 residents)	(+) PE Dx is integrated in the workflow because it is easy to access with one click within Epic. PE Dx is easier than MDCalc because you do not need to open a web browser and type in looking for the risk calculators. If the tool was not as easy to use and that had additional prompts, physicians would not use it.	• Reduced time to complete tasks • Reduced workload
		(−) In order for the physician to change their workflow and switch to using PE Dx, it needs to offer more value and efficiency to integrate in the workflow. ○ PE Dx is not integrated in the workflow as it takes extra clicks to go from the orders to the ED navigator to use PE Dx. ○ Time pressure makes it less likely that the physician will access PE Dx. ○ MDCalc is easier/faster than PE Dx since physicians usually have a web browser open during the shift anyway. ○ It takes the physician extra clicks/time/steps to exit out of the orders/documentation, find the PE Dx within the ED navigator and the return to the orders/documentation to complete their work. ○ Although the PE Dx supports documentation, using PE Dx is a source of extra clicks that the physician does not need in order to determine the order.		• Increased time to complete tasks • Increased workload • Lack of adoption/use
24	Impact on workflow - Adaptation/lack of adaptation to new workflow (6 physicians - 4 attendings, 2 residents)	(+) PE Dx is integrated in the workflow over time after the physician adapts to using it. As more CDS like PE Dx are available, PE Dx will become a part of the habit to use.	• Adaptation/adoption
		(−) When there is time pressure, residents revert to the workflow that is quickest and that they are most comfortable with. Physicians are less likely to use PE Dx and revert to their old workflow.		• Lack of adoption/use
		(−) Physician used PE Dx 2–3 times and then gradually forgot and reverted to old workflow		• Lack of adoption/use
		(−) Physician lack of awareness/habit of using PE Dx is a barrier		• Lack of use
		(−) Physician sometimes does not think to use PE Dx until after placing the order. PE Dx is not integrated in the natural work process		• Lack of adoption/use
		(−) PE Dx is not integrated in the workflow because the physician prefers to dictate or type the note rather than using the documentation support from the CDS.		• Lack of adoption/use
25	Workaround for placing orders via PE Dx versus placing all the other orders (1 physician – attending)	(−) PE Dx is not integrated in the workflow because physicians prefer to place all of their orders for the patient at one time, not just the orders for diagnosing PE. PE Dx supports the D-dimer/CT order but does not fit the workflow of placing all orders together. The physician uses a workaround by using PE Dx and then placing all orders or placing all orders and then using PE Dx to confirm their decision on the order.		• Workaround