Table 3.

Physician responses to optional prompts on user interface. We report the proportion of physicians who were satisfied or suggested improvements when responding to each prompted question. A#’s denote different physicians.

Questionsa	Satisfied	Suggesting improvement	Example responses
PQ1. Do you feel the system speed is tolerable?	4/7 [A5, A6, A8, and A10]	3/7 [A3, A4, and A9]	A3: “I think it’s ok. I mean, some of them, this one was a long one and it's taking a little longer. But I suppose people would probably figure that out.”
			A5: “If there was a progress bar or something it would be okay.”
			A6: “Yeah, it seems really fast today.”
			A9: “I think it takes a little longer than ideal to process it. Well just cause the longer it takes, especially when people are trying to do multiple things. But it’s not too bad.”
PQ2. Do you like the way that the system displays the definitions?	9/10 [A1, A2, A3, A5, A6, A7, A8, A9, and A10]	1/10 [A4]	A3: “I like it fine. I think it’s very easy to see which terms are defined and to see the definition. It’s pretty straightforward.”
			A4: “Because there's an underline, I want to click on it. If you had it blue but not underlined...I don't know what to tell you guys to do, it definitely says to me “click on me” and if I'm moving quickly as people are, like, I would figure it would but it's a little confusing.”
			A5: “Yeah, yeah I do. Just hovering over with the mouse. It is convenient.”
			A7: “Yeah, they're short and easy to read.”
PQ3. Is it easy for you to find the definitions that were generated by the system?	4/4 [A1, A2, A3, and A7]	0/4	A1: “Definitely, I just have to point the mouse at any word that I'm wondering what it means.”
			A7: “Oh yeah, you just hover, that's not hard.”
PQ4. Do you think the instructions on the web page are easy to follow?	7/8 [A1, A2, A3, A5, A7, A8, and A10]	1/8 [A4]	A1: “Yes, very easy”
			A4: “I didn't follow them, I just did what you told me. So, you haven't labeled the search box, so I would...or ‘into the box below.’ That may be confusing, I don't know...I would probably give some direction like ‘push simplify button.’”
			A10: “Yeah, just copy and paste.”

^aPQ# refers to the #th prompted question.