Table 2.

Usability specification and evaluation framework

SDLC stage	Evaluation type	Evaluation goal
Stage 1: Specify needs for setting and users	Type 0: task Type 1: user–task	In the lab or field Describe definition/specifications
Stage 2: System component development	Type 2: system–task Type 3: system–user–task	In the lab—system performance Validity: accuracy, sensitivity and specificity, and speed
Stage 3: Combine components	Type 2: system–task Type 3: system–user–task	In the lab—interaction performance Efficiency: speed and learnability Satisfaction: user perception Validity: accuracy and completeness
Stage 4: Integrate system into setting	Type 2: system–task Type 3: system–user–task Type 4: system–user–task–environment	In the field—quality System effectiveness: accuracy, completeness, utilization, workflow Efficiency: process speed, workflow efficiency Satisfaction: user perception
Stage 5: Routine use	Type 2: system–task Type 3: system–user–task Type 4: system–user–task–environment	In the field—impact System effectiveness: accuracy, completeness, utilization, workflow Satisfaction: user perception Work efficiency: process speed, workflow efficiency Work effectiveness: Practice pattern Prescribing behavior Cost–benefit analysis Quality of care Guideline adherence Patient outcomes Medication errors Communication/collaboration Provider-patient relationship Utilization

The first column indicates system development life cycle (SDLC) stages.

The second column, evaluation type, was added based on Bennett and Shackel's usability model.

Each stage has potential evaluation types that indicate component (user, task, system and environment) interaction in Bennett and Shackel's usability model, such as user–task and system–user–task.

In the last column, evaluation goals represent the expectations for each evaluation type.