User centered design in complex healthcare workflows: the case of care coordination and care management redesign

Abstract

We present the User Need Analysis process for the Integrated Care Coordination Information System (ICCIS) research project at Oregon Health & Science University (OHSU). The primary goal of the process was to select a mix of methodologies to perform user need analysis which would lead to generation of system requirements. The ICCIS Team developers would use these prioritized requirements to build the next generation of the ICCIS software that would help meet the overall goals of the care model. First, semi-structured User Discussion Guides and structured questioners were used to collect raw needs from Physicians, Nurses, IT and Administrative personal. Later Conjoint Analysis (CA) and Dimensional Analysis (DA) were used to generate emerging concepts and priorities from the discussions with the end-user community.

1. Introduction

The Care Management Plus (CMP) program is a validated and disseminated model of care coordination, information technology, and quality improvement in primary care for older adults and patients with complex, chronic diseases. CMP is a continuing cooperative project between the Oregon Health & Science University and The John A. Hartford Foundation.

Although during the program’s previous phases much research was conducted regarding the customer needs and adoption of CMP processes, feedback indicated IT redesign was required; this redesign effort was initiated through grant funding as the Integrated Care Coordination Information System (ICCIS). The development team needed a systematic way of understanding and analyzing missing, incomplete and hidden customer requirements.

A User-Centered new product development process was adopted and the highlights are described in this paper. The process included creating new product objectives, performing a gap analysis on known/unknown user needs, conducting discussions with users and analyzing collected data to understand priorities.

2. Background

The issue of care coordination for patients with multiple chronic illnesses is a significant one for the primary care team. The patients of one primary care physician will see more than 100 separate specialists in the course of one year; for patients with 5 or more chronic illnesses, they will see 13 providers alone (8). Coupled with the increased need for community and social support, the coordination needs – frequently falling on primary care – are enormous. In addition, the knowledge about best practices for chronic illness has been growing; the number of guidelines and protocols that may apply to individual patients defeats the best intention of practitioners to apply; care management through prioritization, patient-centered goal setting, and self-management techniques may help these patients achieve better outcomes.

CMP couples an ambulatory care team with health information technology (HIT). For seniors with complex needs, CMP demonstrated a 20% reduction in mortality, a 24% reduction in hospitalizations and a 15–25% reduction in complications from diabetes.(1,2) CMP facilitates use of HIT to establish and track care plans and specific patient goals, to teach and encourage self-management, to measure and improve quality, and to manage the complex and interleaving tasks as patients and teams prioritize needs. Experience from the dissemination of CMP in more than 75 clinics across the country has led to a deep understanding of the barriers and benefits of such HIT. Barriers include the need to integrate systems, difficulty communicating with the entire team, and representation of workflow.

3. Overview of User-Centered Development Process

Methods for user need analysis were researched to identify a comprehensive mix of the methods that would guarantee user needs are at the center by developing a common understanding of customer needs on the team, identifying ‘hidden’ requirements, ensure that saturation is reached (e.g., requirements are not incomplete), and providing a factual trail for product specifications.

When selecting a user-centered need analysis method it is important to understand the activity within the context of the larger concept development phase. Best practice development processes (3) similar to the process followed for the ICCIS project make an important distinction between customer “Needs” and product “Specifications”. Needs are independent from an actual product and may or may not become part of the specifications for the eventual product the team chooses to pursue (3). Specifications are the items that the development team ultimately implements as part of the final product based on feasibility analysis (3).

Identifying customer needs is an involved process and a five step process is recommended (3):

1. Gather raw data from customers

2. Interpret the raw data in terms of customers needs

3. Organize the needs into a hierarchy of primary, secondary, and (if necessary) tertiary needs

4. Establish the relative importance of the needs

5. Reflect on the results and the process

3.1. ICCIS User Sampling Strategy

Griffin and Hauser found that as a general rule the optimal number of customer interviews is no fewer than 10 and no more than 50 (4). Von Hippel found that it is more efficient to interview a group of customers known as lead users (5). Von Hippel categorized lead users as forward-looking customers who a) struggle with inadequacy of existing products and are in touch with emerging needs; and b) they may have already invented solutions to meet their needs. Often the “customer” is many different people, for example the person making the purchasing decision vs. the person using the system in operations. Burchill recommends using a customer selection matrix for planning interviews (6).

The research team identified two perspectives of users and a set of data collection methods as seen in Table 1. First, a group of every-day users: physicians and nurse care-managers. A second group was the administrative and Information Technology staff that would need to install and support the ICCIS application through its life cycle.

Table 1.

Data vs. Method Gap Analysis

	Type of information needed	Semi-Structured Discussion Guide	Structured Questionnaire
Physician & Nurse Perspectives	Benefit Analysis	X
	Business process Information	X
	Functionality Needs	X
	User Interaction Model	X
IT & Administration Perspectives	Level of IT sophistication		X
	IT development & support experience		X
	Integration with other hospital applications		X
	benefit analysis		X

Physician & Nurse Perspective measures:

• - Benefit Analysis: in what ways does the user believe ICCIS will positively or negatively affect the clinic.

• - Business Process Information: what new workflows or modification are anticipated.

• - Functionality Needs: what features are needed to be a useful product.

• - User Interaction Model: user interface, layout and usability issues.

IT & Administrative Perspective:

• - Benefit Analysis: in what ways does the user believe ICCIS will positively or negatively affect the clinic.

• - Level of IT sophistication: extend of experience with enterprise wide mission critical apps implementation and support.

• - IT Development & Support Experience: programming and customization experience.

• - Integration with Hospital applications: integration with Electronic Health Record, billing, etc.

3.2. Information Gap Analysis & Methodology

In selecting a need analysis method we considered the information we had previously collected from the user community, which included the following: In person observations, Workflow & Requirements Documents from a large clinic, feedback from existing ICCIS prototype users and Site Readiness Assessments. A review of the mentioned documents reviled the gap to better understand functionality and overall IT related needs. Table 1 pulls together the proposed set of collection methods for each specific category of user needs.

4. Daily Users (Physician/Nurses) Discussion Guide

Based on the results of gap analysis, an ICCIS semi-structured discussion guide was designed (available on request). First, we defined the following objectives for the interviews based on the physician and nurse needs: 1) to understand the users’ daily activities, attitudes and values; 2) determine use patterns with current care management products/processes (if any); 3) identify the functional and emotional benefits that the user is seeking from a care management product; and 4) learn about how the usage environment impacts the use and perception of the product. The interview was divided into six sections. Below, we summarize each section’s objectives, structure and discussion flow in order to demonstrate the type of user information extracted in each step.

I. Introduction & general info (15 minutes)

Objectives:

• - Get acquainted.

• - Develop background contextual information.

Structure:

• - Interview with straightforward questions: ‘What does care management mean to you?’)

II. Day in the Life (30 minutes)

Objectives:

• - Understand the context of daily clinic life

• - Discover what role different tools, gadgets and devices play in daily clinic living.

• - Identify the attitudes and values of Physician and Nurses related to activities of daily life.

Structure:

• - Interview / exercise:

  • Part 1: What do you do in a typical day?
  • Part 2: What role do tools, devices and gadgets play in your daily life at the clinic?
  • Part 3: How do you feel about the individual parts of your day?

Discussion Flow:

• While users are telling us what they do during the day, we document activities on Post-Its and paper so we can construct a timeline of their day. After establishing what the activities of the day are, each of the activities or times of the day are probed in greater detail.

III. Process Mapping (30 minutes)

Objectives:

• - To understand the entire case management experience

• - Understand how the process of care management relates to other activities and devices/tools/gadgets/information records

• - Identify use issues, problems, likes/dislikes, and attitudes

Structure:

• - Interview / exercise

• - Begins with verbal questioning then using Post-Its to document the steps in the process worksheet.

Discussion Flow:

• During this part of the interview, we discuss more specifically about managing a candidate patient case. We have the users create a timeline of a positive care coordination care, from identification/referral to a positive outcome, and have them place the post-it notes. Example steps included assessment, care, follow-up, communication, and evaluation; uses of IT systems were also identified with various steps.

IV. Computer Benefit Assessment (20 minutes)

Objectives:

• Identify the primary emphasis and benefits that users are seeking from a care management product.

Structure: 2part, exercise

• - Part 1: Benefit and Feature Review- Have users build their ideal system to identify which features and benefits they choose

• - Part 2: Purchase Scenario- Understand which types of benefits users are most seeking when purchasing a a system.

V. Environment/object audit (10 minutes)

Objectives:

• - Get a glimpse into Physician & Nurses activities by observing evidence of past behavior/decision making through artifacts and office space.

• - Observe the clinic area for interesting adjacencies, peculiarities and work-around.

• - Get a sense of the stylistic expression with which the user identifies

Structure:

• - This exercise takes place throughout the interview/ then at the end of the interview, the researcher may ask to take pictures of additional items.

• - Take notes and drawings of artifacts during interview, including Software Applications Used (even informal uses, like spreadsheets, blank documents), Workspace setup, data collection forms, To do lists (often in paper calendars), and others.

5. IT & Administrative Users Questionnaire

To understand the needs of IT & Administrative users a structured questionnaire was designed. This was designed by reviewing crucial system development and implementation theory, as previously described. The overall interview objectives were: Understand the strategic role of IT in the clinic, determine past success or failure of IT implementation at the clinic, identify IT implementation capabilities of the clinic and learn about impacts of IT at the clinic.

The two part structured IT questionnaire (available on request) had the following objectives:

I. Strategic IT Questions

• - Understand the context and responsibilities of IT within the clinic.

• - Discover perceived IT implementations and usage barriers.

• - Identify clinic’s IT priorities.

• - Learn about Mission Critical needs and relevant past IT support experiences.

II. IT Implementation Questions

• - To understand impact and integration levels with other clinic IT application.

• - Understand healthcare IT standard level of support and implementation at the clinic.

• - Identify IT support contacts for future support activities.

6. ICCIS Data Analysis Strategy

A combination of quantitative and qualitative analytical methods is proposed. Qualitative methods can help customer and clients to think in creative ways about product concepts. Qualitative methods can also be used as input into quantitative methods, for example to identify determinate attributes. Qualitative methods can also challenge assumptions and uncover overlooked perspectives (7).

Conjoint Analysis (CA) is a commonly used tool in concept evaluation and can be used to assess tradeoffs. Conjoint analysis assumes that a product can be represented as set or bundle of attributes. Using conjoint analysis attributes can be grouped together to help identify which sets of attributes would be most preferred by the customer. Attribute Rating Gap Analysis output can be used to select attributes to be used in conjoint analysis.

Dimensional Analysis is performed by listing all the features of a product type. Concept creativity is facilitated by listing all the features and thinking how each can be changed. Dimensional Analysis is most useful when they are long; listing hundred of features is not uncommon. The key to its application is to push beyond the ordinary and visualize dimension that are not seen by other stakeholders (7).

7. Feasibility Study

This section provides details about the feasibility analysis conducted. Respondents similar to ones in the actual study were selected from an internal medicine clinic. Practice interviews were conducted with total of 1 Physician and 3 Nurse Care Manager twice. The IT interviews were tested with one IT Technician from a rural health center.

Overall the discussion guide and questionnaire appeared appropriate. As the result of feasibility study the Physician & Nurses discussion guide went through four revisions before being used in the field. The IT questionnaire went through two revisions. Findings and revision highlights are summarized below:

• - Mapping Care Management Process section in the Nurses Discussion Guide using post-it notes made the respondents critically think about their organizational processes and revealed valuable user data.

• - Evidence of missing requirements was communicated during interviews.

• - Semi-structured nature of the Nurses Discussion guide worked well. Nurses were able to change perspective quickly based on their level of expertise with Care Management Process.

• - Having a guide for discussion conveyed presence of an organized approach and the customer felt their knowledge and schedule was respected, leading to a productive and willingness to participate attitude.

The ICCIS team then identified the top 3 sets of features that were of highest concern to ICCIS study participants:

1. Task Management (free-form, structured)

2. Alarms & Notifications (passive alarms, active alarms)

3. EHR Integration (one-way integration [ICCIS→EHR], two-way integration [ICCIS←→EHR])

There are 8 (2x2x2) different types of software that can be made by combing the list above. Customers can be asked to rank the combinations for preference “like most” to “like least”. Using conjoint analysis the ICCIS team will then identify the customers underlying value system: which attributes are important, and which levels of the important attributes are favored. Table 2 shows what the preference ranking of a hypothetical respondent may be for ICCIS (1 = most preferred, 8 = least preferred). The first three columns are proposed features and each row is an combination of implementation options for the feature set. The last column is a ranking of preference for each combination by one respondent.

Table 2.

Example ICCIS Conjoint Analysis

Then relative importance of attributes can be measured for the individual. Relative importance of each attribute can be expressed for each individual. Adding the utility for each choice we can have an estimate of overall preference.

Having these values allows the ICCIS team to perform a trade-off analysis to identify which attributes are the most important overall and also which levels of these attributes are the most popular. In case the combinations of actual attributes comes out to be too large to survey from each individual, using a reduced set of combinations the ICCIS team can make the user select a set of attributes that most closely meets their desires(7).

8. Conclusion

Many HIT systems and software products available today fail to meet customer expectations and eventually lead to poor adoption rates. A large part of these factors can be attributed to decisions made during the application development process by the research team or vendor. Developers often work in silos and are mostly disengaged from the customer community; as a result the customer’s voice is absent during development and it is not a surprise that so many products fail to meet customer expectations.

Using a User-Centered process for ICCIS we have been able to engage the customer in a meaningful way and as a full partner throughout the development process. First, we were able to systematically look at our available data and compare them to our development Objectives. Once the gaps were realized we formulated a comprehensive data gathering plan.

Later we have optimized product features through conjoint analysis to obtain the greatest common denominator for our users. We continue engaging our users (the clinic) by surveying them for preferences during analysis phase.

Although the methods utilized have root in traditional data collection methods, our process is innovative in the perspective it takes and combination of methods used. We have demonstrated that using a formal new product development process while remaining user centric can be quite informative in the development process.