Table 4.

Usability Session Coding, by physician misses (interventions not acknowledged despite appropriately appearing in scenarios) and successes (interventions leading to desired physician behavior after appropriately appearing in scenarios). (NB: Discharge Pain Prompts can have action rates above 100% because actions were defined as ordering pain medications and updating the pain score – both of which can be prompted by a single intervention.)

Round	1	2	3	1	2	3
CDSS Intervention	Miss Rate	Miss Rate	Miss Rate	Success Rate	Success Rate	Success Rate
1a. 10/10 Pain Assessment Alwert	69.6%	56.7%	0.0%	78.3%	50.0%	45.1%
1b. Pain Re-Assessment Alert	44.4%	41.2%	15.4%	33.3%	70.6%	15.4%
2. Pain Keycept (HPI)	40.0%	0.0%	0.0%	80.0%	54.5%	26.7%
3. One-Click Pain Score Update	n/a	0.0%	0.0%	n/a	41.7%	52.6%
4. Elderly Analgesia Order Set	95.0%	5.3%	16.2%	20.0%	47.4%	41.0%
5. Discharge Pain Prompt	10.0%	0.0%	0.0%	70.0%	120.0%	150%