User-centered design of a scalable, electronic health record-integrated remote symptom monitoring intervention for patients with asthma and providers in primary care

Abstract

Objective

To determine user and electronic health records (EHR) integration requirements for a scalable remote symptom monitoring intervention for asthma patients and their providers.

Methods

Guided by the Non-Adoption, Abandonment, Scale-up, Spread, and Sustainability (NASSS) framework, we conducted a user-centered design process involving English- and Spanish-speaking patients and providers affiliated with an academic medical center. We conducted a secondary analysis of interview transcripts from our prior study, new design sessions with patients and primary care providers (PCPs), and a survey of PCPs. We determined EHR integration requirements as part of the asthma app design and development process.

Results

Analysis of 26 transcripts (21 patients, 5 providers) from the prior study, 21 new design sessions (15 patients, 6 providers), and survey responses from 55 PCPs (71% of 78) identified requirements. Patient-facing requirements included: 1- or 5-item symptom questionnaires each week, depending on asthma control; option to request a callback; ability to enter notes, triggers, and peak flows; and tips pushed via the app prior to a clinic visit. PCP-facing requirements included a clinician-facing dashboard accessible from the EHR and an EHR inbox message preceding the visit. PCP preferences diverged regarding graphical presentations of patient-reported outcomes (PROs). Nurse-facing requirements included callback requests sent as an EHR inbox message. Requirements were consistent for English- and Spanish-speaking patients. EHR integration required use of custom application programming interfaces (APIs).

Conclusion

Using the NASSS framework to guide our user-centered design process, we identified patient and provider requirements for scaling an EHR-integrated remote symptom monitoring intervention in primary care. These requirements met the needs of patients and providers. Additional standards for PRO displays and EHR inbox APIs are needed to facilitate spread.

INTRODUCTION

Roughly 60% of people in the United States have 1 or more chronic diseases. Healthcare for these individuals accounts for 90% of all healthcare expenditures.^1,2 While chronic diseases typically require continuous management, delivery of healthcare is episodic: patients are left to independently manage symptoms between time-limited visits with their providers.³ Digital health tools that are integrated with the electronic health record (EHR)⁴ offer promising and scalable solutions.^5,6 Smartphones, in particular, can serve as a platform for digital health interventions: smartphone adoption is above 81% in the US and higher in other countries.⁷ The increasing number of patients with multiple chronic comorbidities⁸ underscore the urgency to scale and spread digital remote monitoring interventions. The COVID-19 pandemic has heightened this need, as care is increasingly delivered remotely,⁹ particularly for chronically ill, elderly, and other vulnerable patients who are disproportionately impacted.^10–13 Such interventions may prevent emergency room and urgent care visits by facilitating compliance with condition-specific guidelines.

Self-management tools that do not connect patients to their providers face challenges with attracting and retaining users.¹⁴ Integration into clinical workflow via the EHR represents a critical step to enable more robust adoption and is supported by the US Federal Health IT Strategic Plan which calls for the use of “digital engagement technologies beyond portals to connect patients more easily with their providers.”¹⁵ To date, there are few studies of integrated remote symptom monitoring interventions: most have targeted oncology and surgical patients on a standardized care pathway or clinical protocol.¹⁶ Attention deficit hyperactivity disorder¹⁷ and use of PROMIS measures in the ambulatory pediatric settings¹⁸ have also been studied. Furthermore, to be effective and scalable for patients with chronic conditions, clinically integrated interventions require more than summative evaluations: attention to rigorous design based on the real-world needs of patients, caregivers, and providers is paramount.¹⁹ Determining requirements for effective digital health interventions remains a major challenge.²⁰ The lack of attention to the design and development of these interventions may explain the poor track record of broad adoption of digital health innovations into routine care.²¹

We designed a clinically integrated remote symptom monitoring intervention with the intent of broad scale and spread. We selected asthma as a model because of its high prevalence (8% of the US population), disease burden, and healthcare costs, all of which are largely preventable with appropriate management and treatment.^22–25 Additionally, in asthma, clinical guidelines based on clinical trial evidence are well-established and recommend symptom monitoring between visits through patient-reported outcomes (PROs).

As part of our prior feasibility study,^26,27 we developed and tested an initial prototype of this intervention for asthma patients in pulmonary subspecialty clinics to understand clinical workflows and key technology features required for patients with poorly controlled asthma. Our original intervention consisted of a smartphone app that prompted patients to report asthma symptoms every week, an electronic provider view showing the reported PRO data, and application workflows. If patients reported worsening asthma symptoms, the app prompted them to request a phone call from a nurse in their clinic. While the app prototype was not integrated into the EHR, PRO data were manually inserted into physician notes to test the utility of EHR-integrated PRO reporting during the clinic visit. We demonstrated high patient adherence and retention, low provider burden, as well as other benefits.²⁶

For our current study, we adapted this intervention to the primary care setting, where most asthma patients receive routine care.^28–30 In contrast to sub-specialty care, the care of asthma patients affiliated with primary care clinics requires attention to a breadth of health issues, different clinical workflows, variability in asthma knowledge among the care team, and variability in level of disease severity. Within PCP panels, asthma patients are relatively fewer in number, and many have less severe asthma but can still experience exacerbations and benefit from monitoring. Furthermore, by adapting our original intervention for primary care, we envisioned developing a model of a clinically integrated, between-visit remote monitoring intervention that addresses user needs and has potential for broad scale and spread. If successful for asthma, this intervention could be replicated for other diseases (eg, rheumatoid arthritis, depression) and for patients with multiple chronic conditions.^31–34

OBJECTIVE

Our objective was to determine software requirements, defined as “software capabilit[ies] needed by a user to solve a problem or achieve an objective,” ³⁵ for users (patient, primary care provider, nurse) and for integration of third-party applications with the EHR—both of which are necessary to create a remote symptom monitoring intervention that has potential for scale and spread across the primary care setting for asthma patients. For this study, user requirements included functional requirements for the technology (patient app and provider dashboard), workflow needs of nurses and PCPs, and high-level user interface design needs for the purpose of collecting PRO questionnaire data and monitoring asthma symptoms between visits according to clinical guidelines.^22,23 We also determined EHR integration requirements, a type of technical “interface requirement” which describes how a third-party application that satisfies user requirements would need to interact with the EHR.³⁶

MATERIALS AND METHODS

Overview and framework

To elicit user requirements for the intervention, we conducted a secondary analysis of interview transcripts from our prior feasibility study. We then employed user-centered design (UCD) methods: design sessions, surveys, stakeholder meetings, and iterative prototyping.^37,38 To ensure the user requirements could be supported by the technology, we sought to determine EHR integration requirements as part of the design process. We developed the patient-facing mobile app and clinician dashboard while planning to integrate the technology with our institution’s EHR (Epic Systems, Inc.).

To ensure scalability, we used the Non-Adoption, Abandonment, Scale-up, Spread, and Sustainability (NASSS) framework to formulate pertinent research questions.²¹ NASSS is an evidence-based framework specifically designed to go beyond adoption to include scale, spread, and sustainability of health technology innovations. Empirical application of NASSS shows that innovations with more complexity are less likely to be mainstreamed.³⁹ We therefore used NASSS to anticipate potential complexities, and identify the simplest, and thus, most scalable, requirements along each dimension and at each stage of our UCD process (Table 1) and analysis (Table 3). We designed the intervention with the NASSS framework in mind from the beginning and used it throughout.

Table 1.

The Non-Adoption, Abandonment, Scale-up, Spread, and Sustainability (NASSS) framework as applied to an asthma symptom monitoring intervention in primary care

Domain and characteristics of scalable interventions	Pertinent research questions for identifying requirements	Methods used to identify requirements for asthma symptom monitoring intervention in primary care
1. CONDITION: Well-characterized, well-understood, and predictable; few sociocultural factors or comorbidities significantly affect care	Which kinds of symptoms are amenable to between-visit symptom monitoring? What sociocultural factors or comorbidities influence propensity to use intervention?	Determine inclusion/exclusion criteria (eg, exclude patients with COPD) through input from clinical experts Analyze data from prior feasibility study Use UCD to identify relevant factors that affect the intervention (eg, level of asthma control, race/ethnicity, language, health literacy, education)
2. TECHNOLOGY: Direct measurements of medical condition; minimal knowledge or support required to use the technology	Which patient-generated measures most directly measure medical condition? What technology is already used by patients/providers and can be leveraged? What functionalities require minimal training?	Use standard PRO measures Integrate with EHR and clinical workflows Use UCD with patients and providers to ensure intervention works with smartphones and EHRs Consult with usability experts Use simple language with attention to health literacy to ensure technology is as easy to use as possible
3. VALUE PROPOSITION: Clear business case for developer with return on investment (supply-side value); technology is desirable for patients, safe, effective, and affordable	What are potential business models to scale technology? What app functionality will be most effective for a substantial number of patients?	Use standards-based data types and APIs and keep requirements minimal to reduce need for large investment in development and costs to adopt Make intervention desirable for patients (using UCD), feasible to implement (engage stakeholders), and consistent with established evidence-based medical guidelines that recommend symptom monitoring Do not require deployment of a new device or other hardware
4. ADOPTER SYSTEM: Minimal changes required in terms of staff roles or expectations of patients/caregivers	What workflows and functionality would maximize use in the setting of interest (eg, primary care)?	Use UCD and discussions with clinic leadership to ensure workflow and technology fits within existing routines (eg, doesn’t require separate login, callback requests routed to appropriate staff) Design app in English and Spanish
5. ORGANIZATION: Minimal changes required in team interactions or care pathways; minimal work needed to establish shared vision, monitor impact	How do physicians and nurses interact? How are care pathways implemented using the EHR? What protocols exist for making changes to work processes?	Meet with clinic physician and nurse leadership to understand existing team interactions and identify minimal changes needed to implement intervention Work with clinic/hospital leadership to develop or enhance care pathways consistent with existing protocols (eg, nurse-driven asthma triage protocol)
6. WIDER CONTEXT: Financial and regulatory requirements already in place; supportive professional organizations	What kinds of related interventions are already supported by professional bodies? How can the intervention align with anticipated changes in financial incentives?	Ensure intervention is consistent with asthma guidelines (eg, from National Heart Lung and Blood Institute) Work with clinic/hospital leadership to design intervention to help clinics succeed under emerging value-based payment methods (eg, reduce need for emergency visits)
7. ADAPTATION OVER TIME: Strong scope for adapting and embedding the technology as local need or context changes	How can new PROs and other EHR data be added to the app and available at the point of care?	Design technology in modular fashion to allow new PROs to be added and displayed along with other relevant data pending UCD results

Abbreviations: COPD, chronic obstructive pulmonary disease; EHR, electronic health record; PRO, patient-reported outcome; UCD, user-centered design.

Table 3:

User requirements for a scalable remote monitoring intervention in primary care

User requirement	NASSS framework domain applied to clinically integrated intervention
Patient-facing requirements
Either a weekly 5- or 1-item questionnaire depending if symptoms were present in the preceding week with data viewable as a graph	Domain 1. Condition: Adjusting number of items based on symptoms allows for patients with mild intermittent, mild persistent, moderate persistent, or severe persistent asthma to be included, simplifying recruitment in the primary care setting
	Domain 2. Technology: PRO questionnaire directly measures asthma symptom control and requires minimal training
	Domain 4. Adopter System (Patient): 5 or 1-item per week PRO questionnaire is acceptable to patients with poorly controlled (5-item) and well-controlled (1-item) asthma, which also could be completed by the patient’s caregiver
	Domain 6. Wider Context: PRO-based symptoms monitoring is consistent with established clinical guidelines
Patient requested callback from nurse	Domain 4. Adopter System (Patient): An option for patient-requested callback on demand desired by patients to facilitate clinical contact. Automatic callbacks based on PROs alone likely unhelpful to patients and burdensome to providers.
Educational videos and text within app, with reminders after weekly questionnaires	Domain 4. Adopter System (Patient): Reinforces education received during clinic visit.
	Domain 4. Adopter System (PCP): Saves PCPs clinic visit time explaining to patients.
	Domain 6. Wider Context: Content of education promotes established clinical recommendations.
Peak flow values, weekly notes, and triggers entered into app and viewable in the app and EHR	Domain 3. Value Proposition: Strongly desired by some patients with severe asthma. Engages patients by consolidating asthma-related data in a single location.
	Domain 4. Adopter System (Patient): Few patients may use these features, so there should be no expectation for using them.
Previsit tips sent to patients as push notification, includes reminder to take their phone to their PCP appointment and discuss their condition	Domain 3. Value Proposition: Tips highly valued by patients, enhancing engagement by prompting discussion with clinicians about patient responses to PRO questionnaires.
	Domain 4. Adopter System (Patient): Minimal additional task for patients, similar to routine reminders of upcoming scheduled clinic appointments.
PCP-facing requirements
PRO dashboard accessible in 1 click within a patient’s chart, including a link to updated guidelines	Domain 4. Adopter System (PCP): Dashboard accessible from chart requires minimal additional work to view the asthma PRO data.
	Domain 6. Wider Context: Reinforces guidelines by facilitating access to them within EHR in context of asthma PRO data
Previsit EHR inbox reminder sent to inform PCPs that patient is using app	Domain 4. Adopter System (PCP): EHR inbox the most preferred method for reminders of PRO data, compatible with existing workflows
Nurse-facing requirements
Callback requests initiates message to EHR inbox monitored by triage nurses	Domain 2. Technology: Minimal training required
	Domain 4. Adopter System (nurse): Aligned with current nurse roles and workflows (ie, similar to responding to a patient phone call)
Nurse-driven triage protocol available when patient requests callback	Domain 5. Organization: Aligned with existing care pathways, enabling nurses to handle additional clinical situations with minimal additional burden
	Domain 6. Wider Context: Reinforces guidelines by incorporating them into nursing working flow and responsibilities

Abbreviations: EHR, electronic health record; NASSS, nonadoption, abandonment, scale-up, spread, and sustainability; PRO, patient reported outcome.

Setting and participants

Our study was approved by the Institutional Review Boards of Mass General Brigham (MGB) and the RAND Corporation and was performed between January 2019 and November 2020. The study was conducted at 5 primary care clinics affiliated with Brigham Health, an academic health system in Boston, Massachusetts. All clinics were members of the Primary Care Practice-Based Research Network and used a single EHR (Epic Systems, Inc.). Clinics ranged in number of physicians (8 to 37 per clinic), physician clinical effort (part-time vs full-time), patient populations (majority Spanish vs English-speaking), and clinic type (teaching vs non-teaching).

Data collection and analysis

As part of our UCD process, we elicited patient and provider requirements sequentially using the 3 methods described below. Because the requirements for each user type and for EHR integration were interdependent, we considered all of them during each stage of data collection and analysis.

Secondary analysis of exit interviews from prior feasibility study

At the conclusion of the prior feasibility study, we conducted interviews with participating patients and providers about their experience with the intervention.^26,27 For the current study, we re-reviewed transcripts of these interviews to identify new technical requirements and user interface enhancements that would improve upon the original intervention and would be appropriate for primary care. Two research team members independently reviewed transcript excerpts to identify new and relevant features. From these data, we assembled a preliminary list of user requirements. We then assessed each requirement based on the interview data and NASSS domains for inclusion in subsequent design sessions.

Design sessions and stakeholder meetings

We conducted a series of in-person design sessions with patients and providers. We aimed to recruit a diverse range of patients and providers in terms of age, gender, race, and ethnicity. One or 2 research team members conducted each design session, lasting approximately 60 minutes for patients and 30–60 minutes for providers. All design sessions were audio-recorded. Participants were shown wireframes developed using an app prototyping tool (Fluid UI) for user interface design⁴⁰ and were encouraged to “think aloud” to describe their reactions to questions about each app feature.⁴¹ We alternated design sessions with patients and providers to ensure we reconciled both perspectives. For Spanish-speaking patient participants, 1 or 2 native Spanish speakers conducted the design sessions. We aimed to keep the reading level for the patient participants at the 6th grade level and assessed patients’ comprehension during design sessions. We gathered additional input through informal meetings with clinic medical directors, nurses, and 2 nurses in system leadership positions. Provider design sessions and meetings with clinic medical directors included questions about workflow and EHR integration. We met weekly with a user-experience expert to advise on interface considerations. We continued recruiting until we no longer identified any new requirements for features those users considered to be critical (saturation).

Audio recordings were independently reviewed by 2 research team members. We used conventional content analysis methods to identify user reactions which fell into the following categories: specific functionality, usability issues, and suggestions for changes to app wording, functionality, or workflow.⁴² After the team members reviewed the recordings, they recorded a bulleted list of findings within each of these categories in Microsoft Word documents, discussed and resolved any differences considering relevant NASSS domains, and applied changes to the wireframes for future design sessions.

PCP survey

To quantify the scope and prevalence of the broad range of preferences identified in PCP design sessions, we developed a survey for PCPs who treat asthma patients from the 5 study clinics. We drafted survey items based on findings from the design sessions and refined the items through cognitive testing with 3 practicing PCPs. Survey question topics included preferences for reminders that PROs were available in the EHR, graphical presentation of PROs, and anticipated benefits of the intervention. We distributed the survey using REDCap.⁴⁰ PCPs were offered a $25 gift card upon completion. We calculated aggregate results of discrete items. For open-ended questions, 2 research team members independently reviewed and coded all responses into 2 categories: provider notification of asthma data within workflows and visual display of asthma data. The research team members then discussed each response individually and selected representative quotes that explained quantitative survey findings.

Software development, enterprise data service provisioning, and EHR app user interface integration

We used an agile software development approach to translate user interface and EHR integration requirements that emerged from our UCD process to ensure that our application would be delivered within the constraints of our study period and MGB’s emerging governance process for digital health.⁴³ We met regularly with our app developer (First Line Software, Inc.) and MGB’s EHR governance teams to determine the most feasible strategies for implementing our technical and functional requirements. We identified key EHR application programming interfaces (APIs) that were required to pull certain data elements into the application; initiated the enterprise data service provisioning process for requesting APIs; identified graphical user-interface constraints for provider access to third-party applications via the EHR; and finally, iteratively refined the user interface, developed using React Native, an open-source mobile application framework.⁴⁴ To ensure security of the protected health information (PHI) collected by the app, the application was hosted inside the health system’s firewall on a secure server and underwent a rigorous security review as required by our institution’s Research Information Security Office.

RESULTS

We analyzed 26 exit interview transcripts (21 patients, 5 providers) from our prior feasibility study, conducted 21 new design sessions (15 patients, 6 providers), and analyzed results of 55 survey responses from 78 PCP (71% response rate), all of whom were physicians. The demographic characteristics of these participants are summarized in Table 2. Prior feasibility demographic data was reported previously.²⁶

Table 2.

Characteristics of study participants

Design session participants
Patients	N = 15
Age (years)
< 30	2 (13)
31-45	6 (40)
> 45	7 (47)
Ethnicity
Hispanic or Latinx	5 (33)
Not Hispanic or Latinx	10 (67)
Race
White	10 (67)
Non-White Native American, Black or African	5 (33)
American
Gender
Male	4 (27)
Female	11 (73)
Language
Spanish	6 (40)
English	9 (60)
Level of Education
Did not finish high school	1 (7)
High school diploma or GED	1 (7)
Some college but no degree	3 (20)
2-year college degree	3 (20)
4-year college degree or more	7 (47)
PCPs	N = 6
Gender
Male	3 (50)
Female	3 (50)
Number of Years since Residency (years)
< 6	1 (17)
6-15	3 (50)
> 15	2 (33)
PCP survey participants	N = 55
Gender
Male	25 (45)
Female	30 (55)
Number of Years since Residency (years)*
< 6	8 (15)
6-15	17 (31)
> 15	27 (49)
Number of half day primary care sessions per week
<3	28 (51)
>4	27 (49)

∗Incomplete participant values recorded.

Patient-facing, PCP-facing, and nursing-facing requirements informed by the corresponding NASSS domain(s) are described in Table 3. Given the interdependencies and interactions among the 3 types of users, requirements based on each user’s experience were reconciled with the perspectives of other users. For example, requirements for patient-facing components were identified based on input not only from patients but also from PCPs and nurses. We note divergences in requirements within and among the 3 user groups (eg, different preferences among PCPs regarding PRO displays). Specific EHR integration requirements (eg, EHR inbox messaging), were necessary to support certain critical user requirements (eg, callback requests routed to a triage nurse). Many of the requirements were determined by a combination of results from the prior study, design sessions, and survey results.

Patient-facing requirements

Results from the UCD process and application of NASSS translated into requirements for the patient-facing app, which were largely consistent for English and Spanish-speaking patients. Feasibility test data and design sessions suggested strong user support for administering the questionnaires on a weekly basis with 48-hour deadlines (eg, “I think it’s great that you give that amount of time. I just did it, because once it pops up, I just do it just to get it off my… to do list.” [Test Patient exit interview]). However, for patients with well-controlled asthma over several weeks or months, feasibility test and design session participants suggested that completing all 5 questions was too burdensome. Consequently, we designed a modification to allow patients without symptoms during the preceding week to complete a single question, addressing whether patients had any relevant symptoms, instead of the entire 5-item PRO questionnaire. An affirmative response prompted the 5-item PRO questionnaire (Figure 1). Patients participating in design sessions viewed this feature favorably: “I like the option of putting ‘no’ and then you’re not going to answer all these other questions.” (Design Patient)

Figure 1.

Example screenshots of patient-facing app. The app prompts patients through push notifications to answer 1 or 5 questions depending on their previous week’s responses. Patients with poor or worsening asthma control are offered the option to request a callback from a nurse in their clinic.

Secondary analysis of our prior feasibility study data demonstrated support for an app feature allowing patients to request callbacks for deteriorating PROs: “I would always answer ‘yes’ [when the app prompted me to request a call] … after a couple of hours, someone from the staff would call me to see how I was doing and if I needed to go in to see the doctor” (Test Patient exit interview). However, the “mandatory callback” feature, in which patients with poor asthma control are automatically called by a nurse, were perceived as too burdensome to nurses and low-value by patients: “it was just annoying, because I’m dealing with asthma… I’m going to call my doctor. I don’t want you [the app] to call my doctor for me” (Test Patient exit interview). Specifically, patients and the nurse participant from our prior feasibility study suggested removing the mandatory callback requirement and only including the optional feature which allowed patients to decline to request a callback. Therefore, we removed this feature and did not assess it further in design sessions.

Other patient-facing requirements included: ability to record notes (prior feasibility study test patients, design sessions); triggers and peak flow measurements to engage some patients with poor asthma control (prior feasibility study test patients, design sessions); previsit reminders to discuss asthma data with providers at upcoming clinic visits (design sessions); and educational information—videos, brief tips about asthma, and general medical care (design sessions). The latter requirement was also reflected in feedback during design sessions from PCPs who supported empowering patients through education, which could then be reinforced during clinic visits. PCPs provided guidance on high priority education topics for patients to view on the app, including how to use an inhaler and how to distinguish rescue from controller inhalers. Two Spanish-speaking patients in our design sessions requested that the app offered faster response times for requested calls. One suggested allowing the app user to enter a level of urgency for the call. However, this request could not be reconciled with existing clinics workflows for responding to patient queries.

PCP-facing requirements

We identified 2 PCP-facing requirements from design sessions and the PCP survey (Tables 3 and 4). The first requirement was a dashboard containing patients’ asthma data that would be accessible with minimal effort from the patient’s chart in the EHR. An interactive graphical display of PRO data (Figure 2) was iteratively developed with input from PCPs. In this display, survey responses were almost evenly divided regarding preferences for whether higher numbers of the graph should mean better asthma control (47%, eg, “I'm trained to think about asthma in terms of ACT scores, so to me a higher number is better”), or worse asthma control (40%, eg, “I usually think about asthma control in terms of how frequent and severe are symptoms–so the higher [the score] = red = bad–fits my general mindset”).

Table 4.

Results of PCP survey about intervention requirements

	No. (%)
Survey Respondents (PCPs)	55
Preferred method of being reminded of PRO data*
In-basket** message (in Staff	35 (64)
Message folder)
Email to your Partners email	2 (4)
Account
Interruptive (pop up) alert in	17 (31)
Epic (also called BPA)
Have a medical assistant tell me	23 (42)
None
Preferred category of patients about which to be reminded of PRO data (select 1)
All patients with the app (even those with no completed questionnaires, eg, to see non-adherence)	6 (11)
Only patients with some PRO data
Only patients with reported worse asthma control since last visit	16 (29)
Other	27 (49)
Never (I don’t want to be reminded)	4 (7)
	2 (4)
Preferred timing of reminder PRO data (check all that apply)
Day before visit	23 (42)
Day of visit	19 (35)
When I open the patient's chart	33 (60)
Other times	8 (15)
Never (I don't want to receive a reminder)	1 (2)
Preferred graph (select 1)
Graph 1 (higher means better asthma control)	26 (47)
Graph 2 (higher means worse asthma control)	22 (40)
I don’t have a preference	7 (13)
Usefulness of generated text (select 1)
Yes, useful often	25(45)
Yes, useful sometimes	22(40)
No, not useful	5(9)
Not sure	3(5)
Anticipated benefits of intervention (check all that apply)
Improve asthma control	36 (65)
Reduce asthma emergencies (eg, ED visits)	25 (45)
Give patients faster access to care	21 (38)
Make patients feel more connected to their care team	38 (69)
Make patients more aware of their asthma	44 (80)
Give me better data for treatment decisions	42 (76)
No major benefit	2 (4)
Change in interest in participating in digital remote monitoring interventions in the era of COVID-19
I am much more interested/I am more interested	32 (58)
I have the same level of interest	21 (38)
I am less interested/I am much less interested	1 (2)

^*Counts are provided for response of “Would be Nice” or “Want This.” Other response options were “I don’t want this” and “No Preference.”

^**In-basket refers to the EHR inbox.

Figure 2.

Asthma dashboard in Electronic Health Record. All PRO data, notes, and triggers entered by the patient are available with 1 click from the patient’s chart. An auto-generated text summary of the patient’s PRO data and link to current guidelines (developed by an asthma expert based on guidelines and asthma research) is shown. EHR is from © 2021 Epic Systems Corporation, used with permission. Dashboard was developed by the research team.

Design sessions with providers yielded several enhancements to the PRO dashboard: an auto-generated text summary of the patient’s recent asthma data that could be incorporated into notes, and point-of-care access to evidence-based guidelines for asthma management (a statement summarizing asthma control goals and a link to more detailed guidance—the statement was developed by a pulmonologist with expertise in asthma from our institution and incorporated guidelines from the National Heart, Lung, and Blood Institute, the Global Initiative for Asthma, and recent asthma literature). In free-text comments and in the design session, PCPs suggested additional types of EHR data that could be incorporated into the display to make it more useful (eg, current asthma medications and refill data, recent ED visits or hospitalizations, name of asthma specialist treating patient). However, other respondents preferred to access these data through current EHR locations (eg, “I don't need other data. I have a routine for following/checking other data that works for me.”)

The second requirement was previsit reminders: all 6 PCPs in design sessions endorsed the idea of having the app remind patients to bring their phones and medications to their visits. Of the 55 providers who responded to the survey, 53 (96%) wanted to be reminded to check the PROs. Of other options offered, 35 (64%) PCPs preferred to receive an EHR inbox message as a previsit reminder; 23 (42%) preferred in-person reminders from a nurse or medical assistant; and 17 (31%) wanted to receive a clinical decision support alert (ie, “Best Practice Alert”) during the encounter. In written survey comments, 3 PCPs suggested adding the reminder in the “huddle note” (ie, a short note within the daily clinic schedule view in the EHR).

Nurse-facing requirements

We identified 2 requirements for nurses to be able to feasibly respond to callback requests in their existing workflows. First, nurses in all 5 clinics required all patient callback requests to be sent in the form of EHR inbox messages to the individual nurse or nurse pool assigned to the requesting patient and affiliated with the clinic. Because nurses’ workflows were centered around responding to EHR inbox messages, nurse managers suggested that other methods for delivering the requests (eg, email) would not be reliable. Nurses requested that these messages include the patient’s responses to the recent questionnaire, directions for how to access the asthma data within Epic, and other key information to assist with prioritizing the callback request (eg, if they had already called the clinic about their asthma).

Second, nurse leaders suggested that a prespecified nurse-driven clinical protocol designed specifically for asthma would be most effective for triaging callback requests. Otherwise, nurses would default to referring patient-reported issues they identify to the PCP rather than addressing these issues themselves, creating an inefficient workflow.

EHR integration requirements

We identified 3 EHR integration requirements (Figure 3). First, PCPs indicated that the asthma PRO dashboard must be easily accessible from patients’ charts before or during a visit. To address this, we were required to register the third-party application as an “extension.” The PRO dashboard could then be added to the main navigation bar for all PCP users, and individual PCP users also had the option of customizing their patient chart view to include the PRO dashboard as a default tab (Figure 2). When selected, the PRO dashboard application reads credentials of the current PCP user (eg, username) and selected patient (eg, MRN) from the EHR, rendering the appropriate patient’s dashboard in context of the current PCP user and selected patient. The PRO dashboard automatically authenticates the current PCP user using our institution’s Single Sign-On (SSO) process, thereby eliminating a second login step.

Figure 3.

Asthma App Technical Architecture and EHR Integration. Three information flows (A, B, and C) are shown with dark blue, orange, and tan, respectively. (A) PROs and other data entered into the app by the patient are made available to the provider in a dashboard in the EHR along with auto-generated text that can be pasted into a note. PRO data are stored in the app server but currently not written back to the EHR database. The dashboard is accessible to the provider using Single Sign On and does not require a separate login. (B) If symptoms meet certain criteria, the patient will have the option to request a nurse callback. This request is sent through the app server to the appropriate nurse or nurse pool’s EHR inbox. (C) Upcoming visit information is read from the EHR database on a daily basis. This information is fed through the app server, which sends information to both patients and providers before a visit. Patients receive tips for their visit, reminding them, for instance, to bring their phone and discuss their asthma with their providers. Providers receive EHR inbox messages reminding them to check the dashboard for PROs and other information gathered by the app regarding their patients.

Second, a data services API was required to send EHR inbox notifications to nurses and PCPs for patient callback requests and previsit reminders, respectively. Since a standards-based API that supports EHR inbox messaging did not exist, we used a custom API built by our institution that allows a third-party app to send an EHR inbox message to any specified EHR user or pool of users. Third, to send previsit tips to patients and reminders to PCPs, the application required access to visit schedules and the identity of the PCP. While visits schedules and PCP identity could be accessed via Fast Health Interoperability Resources (FHIR) APIs (“Appointment” and “Care Team” resources, respectively), we initially decided to use our institution’s custom APIs (given our software vendor’s experience) while exploring the feasibility of transitioning to FHIR APIs supported by US Core profiles as part of our scale and spread efforts.

DISCUSSION

Using the NASSS framework to guide our UCD process, we identified user and EHR integration requirements for the design of a scalable, remote symptom monitoring intervention for primary care. Patient-facing requirements included allowing patients with no active symptoms to answer only 1 item per week instead of 5; allowing patients to view educational videos and enter notes, triggers, and peak flows; and sending relevant visit-related tips to encourage the patient to discuss their asthma with their PCPs. PCP-facing requirements included allowing the dashboard to be viewed in the EHR with 1 click from the patient’s chart and sending the PCP an EHR inbox message preceding patient visits. Nurse-facing requirements included receiving requests for callbacks via EHR inbox message, and a clinical triage protocol for handling such calls. While we were able to integrate the application’s PRO dashboard into our institution’s EHR for clinician access, requirements for sending EHR inbox messages could not be addressed with current open standards, requiring us to use our institution’s previously developed custom APIs.

By applying research methods in each NASSS domain, we were able to identify requirements that would be pertinent to the potential broad use of our original intervention for asthma in a primary care setting once adapted. We were able to reconcile requirements elicited from English- and Spanish-speaking patients, PCPs, and nurses. Inconsistencies in PCP preferences may reflect variations in PCP workflows identified in prior studies.^45,46 Several preference variations could be addressed through configuration options. For example, most PCPs preferred receiving an EHR inbox message reminder regarding available asthma data, while others preferred an alert or to be told by clinic staff. “Alert fatigue” may be the reason most prefer not to receive an alert.^47–50 PCPs differed regarding the dashboard’s purpose, whether to solely display asthma PROs or to serve as a comprehensive dashboard for all asthma-related data in the EHR. While clinicians may distrust data sourced from third party apps, this may become less of a problem as these users become accustomed to integrated apps. The most striking variation we found was divergent preferences among PCPs for the direction of better or worse asthma control on the dashboard, suggesting the need for more standardized presentations to enable consistent communication among patients and providers.^51–54

Our study addresses a notable gap in prior literature by identifying requirements for a scalable digital remote monitoring intervention for adult asthma patients that may have relevance to other clinical domains. As care is increasingly delivered remotely, which has accelerated due to the COVID-19 pandemic, such requirements are likely to become more important.⁹ By applying NASSS framework domains—especially technology, value proposition, and adopter systems for patients and providers—we were able to prioritize scalability throughout our requirements elicitation process. Our effort is distinct from other reported efforts at developing clinically integrated remote monitoring interventions, which lack prioritization of requirements,⁵⁵ require additional clinical staff, such as care managers, to monitor data,⁵⁶ or require a device (eg, for measuring inhaler usage).⁵⁷ These alternate approaches may encounter scalability issues, such as extra complexity due to prioritizing less important requirements, or costs challenges for using devices where they aren’t necessary. Furthermore, we provide new knowledge regarding how a third-party application can be integrated into an EHR with patient- and provider-facing components to enable the use of PROs for between-visit monitoring, complementing our previous work in the hospital setting.^6,58–61 Even with recent advances, FHIR standards could not address all of our requirements for this application, suggesting a need for additional standards, specifically for sending messages to EHR inboxes.

Limitations

Our study has several limitations. First, our remote monitoring intervention is restricted to a single condition. Additional UCD is required to understand requirements for other medical conditions (eg, mental health, rheumatological diseases). Second, clinical users were affiliated with 1 academic health center that used a single EHR (Epic Systems, Inc.). Requirements may vary for other health systems and EHR vendors. As described in the NASSS framework, organizations with lower capacity to innovate may have more complex requirements. Also, organizations without a clearly defined governance process for integrating third party apps into the EHR will likely encounter challenges. Third, our app is dependent on smartphone ownership and use by patients. Although a growing majority of patients have smartphones, some may have difficulty using them (though many may have a caregiver who uses a smartphone).^7,62–64 Fourth, the PRO data was stored in the app server and not written back to the EHR database. However, institutions are currently considering how to integrate and utilize high-value patient-generated data into the EHR.⁶⁵ Finally, EHR functionality and clinician use of EHRs are evolving. As PRO functionality within native EHRs is increasingly adopted, user requirements for clinicians’ engagement with these data may change, and future validation and replication of our findings is necessary.

CONCLUSION

We determined user and EHR integration requirements for a clinically integrated remote symptom monitoring intervention for asthma using PROs. The requirements reflect the NASSS framework and are designed to scale across primary care. This study demonstrates how third-party apps can be used for PRO-based between-visit monitoring in a real-world clinical setting with the goal of maximizing use, usability, and scalability in parallel with native EHR functionality and patient portal offerings.^{6,58–61,66–68} We identified gaps in current technical and graphical standards that would facilitate spread of this intervention. Although we focused exclusively on asthma, these findings may generalize to other chronic conditions that benefit from routine symptom monitoring using standardized PROs, such as rheumatologic disease, mental health illness, and irritable bowel disease.⁶⁹ Similar requirements elicitation approaches also have the potential to develop scalable interventions for monitoring overall health of patients with multiple chronic conditions, such as captured by global health PROs which measure general physical, mental, and social health.^70–73 With further testing, iterative development, and continued attention to scalability, the rapidly evolving efforts of digital remote monitoring between visits may be achievable at the population level for patients with chronic conditions.

FUNDING

This project was supported by grant number R18HS026432 from the Agency for Healthcare Research and Quality. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality.

AUTHOR CONTRIBUTIONS

Concept and design: RSR, AKD; acquisition of data: RSR, SP1, SP2; analysis and interpretation of data: RSR, SP1, JS, AKD; drafting of the manuscript: RSR, SP1; critical revision of the manuscript for important intellectual content: all authors; obtaining funding: RSR, AKD; administrative, technical, or logistic support: SP1, SP2, JS; and supervision: RSR, AKD.

DATA AVAILABILITY STATEMENT

The data underlying this article cannot be shared publicly due to the privacy of individuals that participated in the study. The data will be shared on reasonable request to the corresponding author.