Translating Research into Agile Development (TRIAD): Development of Electronic Health Record Tools for Primary Care Settings

Abstract

Objectives

This article describes a method for developing electronic health record (EHR) tools for use in primary care settings.

Methods

The “Translating Research into Agile Development” (TRIAD) method relies on the close collaboration of researchers, end users, and development teams. This five-step method for designing a tailored EHR tool includes (1) assessment, observation, and documentation; (2) structured engagement for collaboration and iterative data collection; (3) data distillation; (4) developmental feedback from clinical team members on high-priority EHR needs and input on design prototypes and EHR functionality; and (5) agile scrum sprint cycles for prototype development.

Results

The TRIAD method was used to modify an existing EHR for behavioral health clinicians (BHCs) embedded with primary care teams, called the BH e-Suite. The structured engagement processes stimulated discussions on how best to automate BHC screening tools and provide goal tracking functionality over time. Data distillation procedures rendered technical documents, with information on workflow steps, tasks, and associated challenges. In the developmental feedback phase, BHCs gave input on screening assessments, scoring needs, and other functionality to inform prototype feature development. Six 2-week sprint cycles were conducted to address three domains of prototype development: assessment and documentation needs, information retrieval, and monitoring and tracking. The BH e-Suite tool resulted with eight new EHR features to accommodate BHCs’ needs.

Conclusion

The TRIAD method can be used to develop EHR functionality to address the evolving needs of health professionals in primary care and other settings. The BH e-Suite was developed through TRIAD and was found to be acceptable, easy to use, and improved care delivery during pilot testing. The BH e-Suite was later adopted by OCHIN Inc., which provided the tool to its 640 community health centers. This suggests that the TRIAD method is a promising research and development approach.

Introduction

Electronic health record (EHR) adoption has steadily increased among United States primary care practices,¹ yet care teams report dissatisfaction with EHR usability, design, and functionality.² Practice members engage in extensive customization to increase EHR functionality³ and support emerging primary care work processes like behavioral health care.⁴ When refining an EHR to align with practice changes, the design team is not developing new software, but rather refining an existing product; this refinement is challenging as many system aspects are “hard wired” and difficult to change.

A core aspect of EHR customization includes working to understand the goals, workflows, and sociotechnical systems of the clinical care team to ensure that EHR systems support patient-centered care and clinical care team needs.⁵ Developing EHRs and complex systems using waterfall software development processes is time-intensive; therefore, customizing EHRs to be responsive to the changing needs of health care teams can be a daunting task. Recently, there is growing evidence on harnessing agile scrum software development processes to develop clinical decision support tools for EHRs.^6–8 We employed agile sprint development principles to customize the functionality of an EHR to meet the needs of practices that were integrating behavioral and medical care.^9,10

Integrated behavioral health¹¹ in primary care settings is an emerging model of whole-person primary care. Behavioral health clinicians (BHCs) work closely with primary care teams to care for adult patients with mild and moderate behavioral health needs (e.g., depression) and have shown to affect management of chronic care issues (e.g., diabetes). BHC integration involves changes to staffing and scheduling patterns,¹² office space,¹⁰ workflows, and staff responsibilities.⁹ The documentation, information-sharing behaviors, and needs of the integrated care team also change,¹³ and EHRs are not designed to support the new tasks and workflows of integrated care delivery.⁴

Objectives

The purpose of this article is to describe our approach and methods to develop EHR-based tools for primary care settings.

Methods

Conceptual Framework and Design

Recognizing the complexity of EHR modification, we used the Systems Engineering Initiative for Patient Safety theoretical model (SEIPS 2.0)⁵ to guide our assessment of the integrated primary care work system and processes. We combined the SEIPS 2.0-informed system and process assessment with a rapid prototyping and development process that is characterized by: structured engagement between the research and development teams during data collection; data-distillation processes to translate findings into visual displays of concrete workflows, wireframes, and task analysis lists; user-based design activities with BHCs to solicit feedback on high-priority EHR needs and prototypes; and modified agile scrum software development processes,¹⁴ using scrum staffing roles (product owner, scrum master, and a cross-functional scrum team), and sprint cycles.

Setting and Sample

“Turning EHRs into Assets for Mental Health and Uniting Practice” (TEAM-UP) is a research study funded by the National Institute of Mental Health (1 R34 MH 100371). TEAM-UP is a collaboration between researchers in the Department of Family Medicine at Oregon Health & Science University, the University of Colorado–Denver, and OCHIN, Inc. OCHIN is a nonprofit health care innovation center that supports the largest network of United States-based safety net practices using the same Epic EHR.¹⁵ OCHIN has more than 500 community health centers (CHCs) in its network and serves approximately 2.8 million patients.

Six CHCs were purposefully selected based on size, location, and clinics’ readiness to codevelop and implement a behavioral health module for their Epic System. Each CHC was required to have an integrated BHC and use OCHIN Epic. The development team was composed of study researchers from Oregon Health & Science University, software developers from OCHIN Inc., and staff at participating clinics (BHCs and Epic site specialists), and other key stakeholders. The Institutional Review Board at Oregon Health & Science University approved this study protocol (#9366).

Translating Research into Agile Development Process

Assessment, Observation, and Documentation of BHC EHR Usage

Table 1 highlights the methods we used to assess, observe, and document BHC EHR usage. The SEIPS 2.0 model provided a framework for capturing the six interacting components in the work system and two factors in processes domain. The first column in Table 1 lists these components: external environment, internal environment, organization, tools and technology, task factors (e.g., tasks BHCs performed in integrated care teams), person characteristics, professional work processes, and collaborative work processes. The methods we used to capture this data are listed across the first row of the table. We conducted presite visit interviews, observation, informal and semistructured interviews, and observation of EHR documentation. The presite visit interview included questions related to the internal clinical environment (technical infrastructure, clinic layout, and organization of teams, etc.). Presite interview data were used to develop the observation guide used during site visits.

Table 1.

Assessment and observation of BHC EHR usage

SEIPS 2.0 framework elements	Presite visit interview	Observations	Informal and semistructured interviews	Observation of EHR documentation
External environment Macrolevel factors that impact healthcare processes (e.g., policy, economics)			✓
Internal environment Features of internal environment affecting care delivery and EHR use (e.g., technical infrastructure, physical space and clinic layout for BHC and primary care team, computer use by BHC, characteristics of staff and staff roles in integrated care delivery)	✓	✓	✓
Organization Training and background of staff and providers, characteristics of work schedules and assignments, management and incentive systems, such as payment for BHCs, BHC training, documenting policies, resource availability, and culture		✓
Tools and technology Objects that integrated teams use to do work or that assist people in doing work. (e.g., information technologies, medical devices, physical tools, equipment)		✓	✓	✓
Tasks Attributes or characteristics of the task, such as difficulty, complexity, variety, ambiguity, and sequence		✓	✓	✓
Person / person factors within CHCs How teams complete their tasks; knowledge and perceptions of HIT and demographic characteristics of team members (e.g., gender, age)		✓	✓
Professional work Tasks and individuals who performed these tasks; under ideal circumstances, what information did users need to carry out each task related to integrating patient care, and the ways in which needs were met / unmet when working on behavioral health issues		✓	✓
Collaborative professional-patient work Work in which both health care professionals and patients (and/or family) are jointly and actively involved (e.g., development / modification / execution of care plan, identifying, support and tracking accomplishment of self-management goals)		✓	✓

Abbreviations: BHC, behavioral health clinician; CHC, community health center; EHR, electronic health record; HIT, health information technology.

At each clinic site visit, we conducted in-depth observations, informal and semistructured interviews, with a range of clinical team members, to identify tools and technology, tasks, person factors, and work processes associated with integrated care processes. Length of the site visits were 2 and 4 days, depending on practice size. Our qualitative data from site visits were used to tailor the semistructured interview guides to both the practice and interviewee. We conducted the semistructured interviews (n = 27) with two to four practice members at each clinic to understand the health information technology (HIT) needs and workflows of integrated care teams. Interviews lasted 1 hour. Finally, we asked BHCs to provide real-time input about gaps between workflows and EHR functionality arising naturally in their daily EHR use workflow.

Structured Engagement of Research and Development Teams

To develop the EHR tool, two members of the field research team worked closely with a development team that included an OCHIN software developer, Epic site specialists from two of the clinics, and a clinical informatics expert. This team’s work was periodically informed by two consultants (human factors expert and clinical psychologist).

Throughout the project we structured interaction opportunities for the field research team, development team, and consultants. The field research team needed to share with developers what it was seeing and hearing in practices, so that these grounded insights could inform EHR development —without requiring the development team to read lengthy qualitative field notes or semistructured interview transcripts. We scheduled in-person meetings with team members (field research, development, and consultants) to debrief from clinic visits and discuss emerging findings. Conversations served as brainstorming sessions to identify issues needing more field investigation, and stimulated ideas for development.

Data Distillation

The field team developed artifacts of each practice and included key workflows of BHCs and integrated teams, workflow roles, subprocesses in the workflows, tasks for each step, and associated EHR challenges. To depict potential EHR solutions, the team then developed wireframes, using Pencil, an open source graphical user interface prototyping tool (https://pencil.evolus.vn/). Wireframes are visual representations of the functions and rules for how information is displayed.¹⁶

Developmental Feedback from Clinical Team Members

The field research team used the products from the data distillation phase to facilitate dialogue with clinical team members and to refine EHR solutions prior to development in OCHIN Epic. To do so, we identified a small group of BHCs—approximately five from different practices—and held a series of video-conferenced developmental design meetings.

Agile Scrum Sprint Cycles for HIT Prototype Development

OCHIN developers used the wireframes to create a β version of the BH e-Suite tool using agile scrum development principles.¹⁴ We assigned the role of product owner to a clinical informaticist at OCHIN, and our lead qualitative researcher attended trainings to become an official scrum master. An Epic specialist and a clinical informaticist (product owner) comprised the functional development team. We convened six 2-week sprint development cycles during June to August, 2014, addressing three domains of EHR modification: (1) assessment and documentation, (2) information retrieval, and (3) monitoring and tracking.

Results

Assessment, Observation, and Documentation of BHC EHR Usage

BHCs used the primary care EHR for their documentation and communication needs. The EHR was not tailored to the BHC’s EHR needs and therefore lacked the functionality necessary to support their role of integrating behavioral health in primary care clinics. From our multivaried data collection efforts, we noticed that BHCs needed additional features to support their role in delivering behavioral health screenings, documenting behavioral health history, rapidly accessing patients’ relevant social and medical history, and tracking patients’ progress on goals.

Engagement of Research and Development Teams

We refined our data collection approach iteratively, in response to developer feedback, to focus observations and interview guides. Our goal was to obtain meaningful descriptions of BHC workflows, tasks, and technical needs, including administrative sequelae that would need to be addressed, such as the inclusion level of service codes. The developers explored possible solutions to arising field team needs, including gaining permission to adapt paper-based screenings to an electronic environment and exploring best methods for documenting and tracking patient goals, etc.

Data Distillation

Table 2 is an example of a task analysis showing the workflow steps, tasks, and technical challenges or EHR gaps that could inform development. This list informed the brainstorming of the field team and developers and led to the identification of potential development areas and solutions.

Table 2.

Task analysis

Workflow step	Tasks	EHR challenges
Patient arrival	Check-in patient Look for health maintenance alert Hand out screener, if appropriate	Alert for annual screener overlooked or forgotten
Need identification	Asking patients questions Validated screeners Calculating scores Primary care clinician’s (PCC) subjective observation of problem Entering score in assessment section Enter in PCC chart note	Paper scores hand-entered in Epic in Assessments tab Primary care progress note body Mistakes inherent in provider identification of need and standardized screeners Entering scores takes time Scores entered in multiple places Score in chart note easy to overlook
Introduction to BHC	Warm hand off or referral	Difficulty in accessing BHC Referral reason not always entered
Chart scrub	Review patient history, medications, past visits, laboratories, referral notes to prepare for visit	Information can be buried in chart notes BHC doesn’t have timeto find most important pieces No referral notes
Encounter	Progress note documentation of patient encounter Goal setting and plans for return Make next appointment	Free text makes it difficult to use data for quality improvement (QI) Templates are nonstandardized Vocabulary in Epic not suited to BH Typing in note free text takes time
Diagnosis	Formulate clinical diagnosis	Not personalized for BH diagnosis (dx) Free typing in dx takes time
Close note	Close chart Route to patient’s PCC	No way to easily or systematically close loop with PCC or patient

Abbreviations: BHC, behavioral health clinician; EHR, electronic health record.

We drafted wireframe diagrams that were intentionally rough to encourage user feedback, as highly refined pictures would have been “too finished,” discouraging input. Fig. 1 shows a wireframe designed for the “need identification” domain, to illustrate the need for automated screening assessments used by BHCs (e.g., Patient Health Questionnaire-2). The boxes on the left side of the figure (e.g., chief complaint, history) reflect the action buttons BHCs wanted for quickly accessing other commonly used parts of the record. The area on the right shows what the user might see if they clicked on “BH assessment.” Several other buttons would bring up patient assessment tools. The bottom portion reflects work to auto-populate a progress note; this was ultimately achieved differently than shown here.

Fig. 1.

Automated screening wireframe for behavioral health clinicians.

Developmental Feedback from Clinical Team Members

We reviewed with the BHCs the list of technical challenges, using Table 2 as a checklist; one high-priority need was more efficient EHR-based screening. Paper-based screening for behavioral health was common yet cumbersome, and BHCs entered scores on paper and then either manually entered them into the EHR or had it scanned and entered into the chart as a PDF. Automated screening and scoring was the first area for development; the field and development teams reviewed existing OCHIN Epic functionality and identified templates that could be repurposed to address BHC assessment and scoring needs. Wireframe mock-ups demonstrated the design and proposed functionality of the screening tools; for example, values could be entered into the EHR template, EHRs would tabulate scores, and track scores over time. BHCs provided feedback multiple times until they were satisfied with the design.

Agile Scrum Sprint Cycles for HIT Prototype Development

Developers put the first iteration of the tool into the Epic test environment. EHR walkthrough meetings were held with BHC “super users” to ensure they had a deep understanding of the tool and the intended uses and features. The product owner (OCHIN clinical informaticist) facilitated meetings, and BHCs tested this build for 3 weeks, clicking all buttons, testing use case scenarios, etc. In parallel, a PDF version of the tool was shared with the consultants for review and feedback; user and expert feedback informed additional, minor refinements to the tool. Subsequently, quality assurance testing was conducted by OCHIN prior to its release, as the final step before tool implementation and pilot testing, that is, asking BHCs to use the tool in everyday practice.

BH e-Suite Tool

The BH e-Suite resulted out of our use of the Translating Research into Agile Development (TRIAD) approach to modify an existing EHR into a tool specifically for BHCs embedded with primary care teams. To meet their clinical and workflow needs, BHCs required assessment and documentation, information retrieval, monitoring and tracking functionality. To this end, we developed six features for assessment and documentation (BH automated assessment tab, visit diagnosis dropdown lists, progress note, speed buttons configured with BHC level of service codes), two features for information retrieval, and four features for monitoring and tracking needs. Table 3–¹⁷ shows the domains in which HIT challenges were observed, the feature developed in response to these challenges, and a description of the e-Suite features.

Table 3.

BHCs’ health IT challenges and BH e-Suite features developed

BHC health IT challenge/Need	Feature developed	Description of new BH e-Suite feature
Assessment and documentation needs
Reduce use of paper screening tools	BH assessment tab	Dropdown lists and templates of common BHC screeners build into EHR
Eliminate manual entry and tabulation of scores		Templates auto-tabulate scores
Track scores over time		Screening scores displayed over time (date of screening noted) with an indication of risk level
Quickly select diagnosis codes and indicate the severity	Visit diagnosis	A dropdown list of diagnosis tailored to the BHC Diagnosis can be annotated as mild, moderate, or severe
Quickly document patient progress during appointments	Progress note	Speed buttons and free-text comment fields provided on assessment page for quick documentation All of the features described above auto-populated progress note; BHC retain the ability to add free-text notes, as needed
Populate eligible BH billing codes for successful reimbursement	Level of service (LOS)	Speed buttons configured with LOS codes used by BHCs for billing purposes; LOS codes customized to include billable BHC encounters (Oregon only)
Information retrieval needs
Quickly identify personal and family history related to BH	(1) Personal and family BH history (2) Snapshot	Prepopulate family history tailored for BHCs with commonly used information such as history of anxiety, depression, substance use, diabetes, high cholesterol, and blood pressure
Quickly review BH history during a visit	(1)	Prepopulate patient social history using frequently used terms, including head trauma, insomnia, sexual abuse, and trauma Provide a link to medical history for quick review Summary section populated with BH-relevant information (BH history, diagnosis, and record of BH screening scores)
Monitoring and tracking needs
Mechanism to easily document goals	Goals, challenges/opportunities and follow-up	Table created to document goals with a dropdown list of common self-management goals
Document/review patients challenges and opportunities to change/achieve goals		Template developed to quickly document/review common strengths and social barriers and comment box for elaboration on details
Track patient goals over time		The summary section that provides a snapshot of the patient goals over time
Track patient panel for follow-up and outreach	Repurposed existing space and tools -Reporting workbench	Lists details to review for the patient’s next visit Communicate scheduling instructions for Front Desk staff Ability to forward the encounter with a message to another clinical staff member Reminders function created in In Basket; BHC uses this to recall specific tasks.

Abbreviations: BHC, behavioral health clinician; EHR, electronic health record; IT, information technology.

Note: This table is reproduced with permissions from the Journal of Innovation in Health Informatics.

Discussion

In the United States, health care is still in the early phase of developing sophisticated technologies to integrate innovative patient care models and clinical workflows, suggesting that the field is open to many new avenues and applications.¹⁸ EHRs, initially configured locally at many organizations primarily to facilitate billing processes, are now being customized particularly by academic health centers, to support the changing environment of health care.¹⁹ While our application of the TRIAD method focused on the use case of BHCs in integrated care settings, our development process could also be applied to other health care uses such as for the collection of patients’ contextual information (e.g., social determinants of health).²⁰

When refining an EHR tool to align with a practice change, because the design team is modifying an existing product rather than developing new software, challenges arise. One critical aspect is that some EHR systems are extensible—extended functionality can be implemented without impacting the internal structure of the systems—and others are not. The Epic EHR is one such extensible system and is thus open to customization experiments.¹⁹

TRIAD combines mixed-methods research with user-testing and agile scrum software development processes to expand and customize EHR functionality for health care teams’ emergent HIT needs. The TRIAD method led to the development of the BH e-Suite, which was found to be acceptable and easy to use by BHCs in subsequent pilot testing, and improved care delivery.²¹ This tool was subsequently adopted by OCHIN Epic for release to OCHIN’s 640 CHCs, suggesting that the TRIAD method is promising as a research and development approach.

Our study has two important considerations for EHR vendors. First, EHR vendors have different organizational capacities to engage in rigorous user-testing processes, often citing development timelines as an impediment.²² We spent 3 years on the development of BH e-Suite, including mixed-methods data collection, analysis, and tool development, while engaging in user testing and pilot testing. For non-researchers and practice members not trained in qualitative data analysis, the development timeline could be abbreviated by conducting a less rigorous analysis, although an obvious development challenge would be to create a robust process for prioritizing practice members’ HIT needs. To create efficiencies and to limit cost in this type of work, we recommend engaging a small number of practices and purposively selecting team members who are engaged in both data collection and the developmental phase of the work. Second, our study developed EHR functionality in response to the changing environment of primary care. Emerging care models such as Accountable Care Organizations²³ illustrate that the EHR needs of practices will continue to evolve, and EHR vendors must build their capacity to be responsive to these changes. Two critical organizational prerequisites of any organizational change activities regarding behavioral health integration and EHR customization include leadership practices²⁴ that support the work and financing of the effort. OCHIN is an Epic vendor for CHCs. As part of their business model and approach for serving the needs of CHCs, they employ developers and informatics experts that work to optimize the EHR for their clients. Thus, the technical expertise to develop EHR tools exists in this organization, as it does in other vendor organizations.

The results of this study should be considered in light of one limitation. Our TRIAD method was used to generate modifications to the OCHIN Epic EHR. While the functionality we developed may be unique to Epic and limited in terms of generalizability, we believe that the methods we used outline a basic process for assessing the HIT needs of various end users in primary care.

Conclusions

The TRIAD method is an approach that primary care teams or researchers can use to rapidly assess local practice workflows, tasks, and information needs and translate these needs into EHR tools that are acceptable, easy to use, and improve clinicians’ and clinical teams’ delivery of care.