The ESCAPE Framework: A Practical Two-Phase Mixed-Methods Approach for Usability Evaluation of Interoperable Health IT

Abstract

Persistent usability challenges in health information technology (health IT) contribute to documentation burden, workflow inefficiency, and clinician burnout, particularly for interoperable tools such as clinical decision support (CDS) systems and mobile health (mHealth) applications that are developed as stand-alone solutions and later integrated with electronic health record (EHR) systems. Poor usability can disrupt data flow, hinder workflow integration, and undermine interoperability goals. We developed the Efficient Strategy for Comprehensively Assessing (and fixing) Problems Early (ESCAPE) framework as a practical, theory-driven, two-phase mixed-methods approach for usability evaluation before and after EHR interfacing. Grounded in human–computer interaction and implementation science principles, ESCAPE integrates expert and end-user evaluations using heuristic evaluation, task-based think-aloud protocols, perceived-usability surveys, and cross-phase focus groups to support iterative refinement and readiness for adoption of interoperable health IT tools.

1. Introduction

Usability remains a persistent challenge in health information technology (IT). Poorly designed systems contribute to documentation burden, workflow inefficiency, and clinician burnout, compromising data quality, decision support, and patient safety [1–6]. Usability—the extent to which a system enables users to achieve goals effectively, efficiently, and with satisfaction [7]—is often evaluated too late or with a single method.

Interoperable tools such as clinical decision support (CDS) systems and mobile health (mHealth) applications, typically developed as stand-alone solutions and later integrated with electronic health record (EHR) systems, introduce additional complexity. Poor usability in such tools can disrupt interoperability, reduce clinician trust, and compromise adoption [8].

To address these gaps, we developed the Efficient Strategy for Comprehensively Assessing (and fixing) Problems Early (ESCAPE) framework, a structured, replicable process for usability evaluation across the development lifecycle. Grounded in human–computer interaction and implementation science principles that emphasize contextual fit, stakeholder engagement, iterative refinement, and readiness for adoption, ESCAPE focuses on both usability and implementability. This paper presents the conceptual foundations, structure, and application of ESCAPE as a two-phase mixed-methods framework for evaluating and enhancing the usability of interoperable health IT tools.

2. Conceptual Foundations

ESCAPE is grounded in established theories of health IT usability and technology adoption. First, it adopts the stratified view of health IT usability, which conceptualizes usability across the system/interface, user–task interaction, and socio-technical context, ensuring that expert, end-user, contextual perspectives are represented across both phases of evaluation [9]. Second, ESCAPE operationalizes core usability constructs from the Health Information Technology Usability Evaluation Model (Health-ITUEM)—effectiveness, efficiency, and satisfaction— using quantitative task-performance metrics, validated perceived-usability surveys, and qualitative think-aloud data [10]. Third, ESCAPE incorporates behavioral determinants from the Unified Theory of Acceptance and Use of Technology (UTAUT), including performance expectancy, effort expectancy, and facilitating conditions to contextualize usability findings in relation to adoption and implementation readiness [11].

Unlike traditional formative or summative usability testing approaches often conducted as single-phase evaluations, ESCAPE explicitly structures usability assessment across two development stages: before and after EHR interfacing. Whereas standards such as IEC 62366 and many CDS usability evaluations emphasize predeployment risk mitigation or interface-level assessment, ESCAPE extends evaluation into the integration phase, where interoperability-related workflow disruptions frequently emerge. This phased, traceable structure enables systematic comparison of usability outcomes across development milestones rather than isolated assessments.

3. Methods Overview: The ESCAPE Framework

3.1. Overview

ESCAPE is a two-phase mixed-methods framework designed to identify, prioritize, and refine usability issues before and after EHR integration (Figure 1). Each phase includes (a) expert testing (heuristic evaluation) and (b) end-user testing (task-based think-aloud protocol + perceived-usability survey), with (c) cross-phase focus-group discussions to compare findings. Standardized usability artifacts—Problem Log, Design Change Log, Scenario Pack, and Measurement Codebook—ensure comparability and traceability of refinements.

Figure 1.

The ESCAPE Framework.

Sample size: Five experts are typically sufficient to identify most usability issues [12], while twenty end-users identify ≈ 95% of usability problems [13]. Participants may overlap across phases to enable direct comparison of perceived usability following EHR interfacing, or differ to reduce recall bias, depending on study constraints. ESCAPE applies sequential iteration across phases as findings from Phase I inform refinement prior to EHR integration, while Phase II evaluates usability changes introduced by integration, supporting cumulative improvement across the development lifecycle.

3.2. Phase I — Before EHR Interfacing (Stand-Alone Builds)

1. Expert Testing (Heuristic Evaluation): Five usability experts trained in human– computer interaction and health informatics independently review representative use-case scenarios against Nielsen’s heuristic principles [14]. Each expert rates the severity of identified issues (0–4) using a standardized checklist. The research team consolidates overlapping findings and finalizes severity rankings, producing a prioritized Problem Log and Design Change Log. Identified usability issues are refined and corrected prior to end-user testing.

2. End-User Testing (Task-Based Think-Aloud Protocol + Perceived-Usability Survey): Twenty target end-users perform workflow-based use-case scenarios in a simulation environment using a concurrent think-aloud protocol. Data include screen recordings, task-completion rates, time on task, error counts, and coded qualitative observations on ease of use and usefulness. After testing, they complete validated perceived-usability surveys to capture subjective perceptions of effectiveness, efficiency, and satisfaction. Quantitative and qualitative results are triangulated to identify refinements needed before EHR interfacing.

3.3. Phase II — After EHR Interfacing (Interfaced Builds)

1. Expert Testing (Heuristic Evaluation): Five usability experts apply the same heuristic-evaluation procedure to the interfaced version within a sandbox EHR environment, emphasizing workflow integration, data exchange, and adherence to EHR conventions. Workflow integration is assessed by examining how seamlessly the tool fits within clinical task sequences (e.g., minimizing context switching between modules, maintaining consistency with established order-entry workflows). Results inform a second round of refinements documented in the Design Change Log.

2. End-User Testing (Task-Based Think-Aloud Protocol + Perceived-Usability Survey): Twenty end-users who meet the same inclusion criteria as those in Phase I execute the same use-case scenarios within the EHR context. Integration-specific steps (e.g., authentication, data persistence, context switching, supporting role-specific actions) are observed and analyzed. Perceived-usability surveys are completed, and results are compared with those from Phase I to assess whether integration improved or degraded usability.

3. Cross-Phase Focus Group Discussion: Following Phase II, representative end-users and experts participate in semi-structured focus group discussions to compare experiences between the stand-alone and integrated versions. Discussions explore how system refinements affected workflow, performance, and perceived usability, providing interpretive insights for future design and validation.

Meaningful usability improvement across phases may be evidenced by reductions in heuristic severity scores, improved task-completion rates and efficiency metrics, decreased error frequency, and positive shifts in perceived-usability survey scores. ESCAPE emphasizes relative improvement and cross-phase comparison rather than absolute thresholds, supporting context-sensitive interpretation across tools and settings.

4. Applications and Replicability

ESCAPE is adaptable across a range of interoperable health IT tools intended for EHR integration. Phase I enables early identification and correction of interface- and workflow-level usability issues, minimizing costly redesign, while Phase II captures integration-specific challenges related to workflow fit, data exchange, and system conventions. For example, in a CDS tool supporting risk stratification, Phase I evaluation identified unclear terminology, redundant data entry, and navigation inefficiencies, whereas Phase II testing revealed integration-specific issues such as context switching, alert timing, and data persistence that were not observable in the stand-alone build. Standardized artifacts—including the Problem Log, Design Change Log, Scenario Pack, and Measurement Codebook—support traceable refinement and comparability across phases.

5. Discussion

Although usability testing is widely recommended, it is most often conducted either early in development or immediately prior to deployment. Few frameworks explicitly distinguish usability challenges arising from stand-alone design versus those introduced by EHR integration, despite evidence that late-stage interoperability failures are a major cause of adoption breakdown. ESCAPE addresses this gap by formalizing usability evaluation across two phases aligned with integration milestones, advancing usability science through a structured, iterative process rather than a single testing event.

Structured usability evaluation is particularly critical for CDS systems and mHealth applications, which often originate outside enterprise EHR environments. During integration, usability issues such as mismatched data elements, inconsistent interface logic, or workflow interruptions may emerge [15,16]. ESCAPE provides a systematic approach to detect and resolve these issues, ensuring that interoperability enhances rather than hinders usability and clinical performance.

ESCAPE bridges formative and summative usability evaluation by supporting continuous refinement via heuristic evaluation, think-aloud protocols, perceived-usability surveys, and cross-phase focus groups. While adaptable to settings with varying resources, it requires representative users and, for Phase II, an EHR sandbox environment, which may limit applicability in some contexts. Repeated testing may increase participant burden and require careful coordination. Future work should validate ESCAPE across multiple institutions and EHR platforms, benchmark outcomes against single-phase usability approaches, and explore alignment with regulatory, certification, or procurement processes for interoperable health IT.

6. Conclusion

The ESCAPE framework provides a scientifically grounded, replicable two-phase mixed-methods approach for identifying and improving usability before and after EHR interfacing. By integrating human-factors principles and behavioral theory within an iterative structure, ESCAPE enhances the rigor and practicality of usability science. Its flexible design supports comprehensive evaluation of interoperable health IT tools, ultimately promoting safer, more efficient, and user-centered digital health innovation.