Parent‐Reported Usability of a Patient Portal‐Based Asthma Care Tool for Parents of Children With Asthma

ABSTRACT

Introduction

This study evaluates our new EHR‐integrated patient portal for asthma care (PAC) management module for parents of children with asthma. The module includes a previsit asthma intake questionnaire via the portal. The parent answers are integrated into the provider's clinic progress note to support clinical decision‐making. Our goals were to measure the functionality and usability of the PAC module and to understand facilitators and barriers to its use for parents.

Methods

Parents of children ages 0–11 years old (n = 45) completed the PAC module's asthma intake questionnaires prior to their upcoming pediatric pulmonology clinic visit. To assess functionality, provider progress notes were manually reviewed to measure the amount of key asthma‐related data captured. Differences in percent data captured with and without the PAC module were compared. Electronic surveys capture demographics, usability data (the System Usability Scale [SUS]), and open‐ended experiential feedback about the module. Analysis included descriptive statistics for demographics and usability, as well as the constant comparative method for open‐ended feedback.

Results

The PAC module at this early stage of design significantly improved the capture of key asthma data in physician notes, increasing from 77% to 92% (p < 0.001). The average SUS score (83.8) indicated high usability. Favorable aspects of the module that were identified included time savings and ease of use.

Conclusion

Our PAC module enhanced data capture of key asthma management elements and demonstrated high parental usability. We will continue to refine the module through an iterative approach based on end‐user feedback, with future expansion planned for broader patient populations.

1. Introduction

Pediatric asthma management is complex due in part to the multitude of factors that contribute to symptoms and the existence of multiple evolving asthma guidelines [1, 2]. The assessment of patient symptoms and functioning over a specified prior time period is usually accomplished during healthcare visits. However, these healthcare visits have become shorter, leaving less time for providers to obtain key asthma history data from patients to drive informed clinical care decisions. In addition, there is less time to spend with patients discussing the care plan, which can result in patients who are less informed about their underlying condition and its management. To address these systemic constraints, novel approaches using technology are being integrated into practice to help facilitate data gathering, decision‐making, and patient involvement.

One technological approach for this is electronic health record (EHR)‐integrated patient portals. Electronic patient portals are online secure websites that give patients and their parents access to their personal health data at any time, provided the user has an internet connection [3]. Several features of the patient portal are advantageous over other modalities, including the ability to communicate with providers, share patient educational materials, refill prescriptions, and make appointments. The portal, especially when integrated with the EHR, advances patients' agency surrounding their own healthcare management [4]. Patient portals can also assist in gathering data from patients (i.e., patient‐reported outcomes) before a face‐to‐face or telehealth visit, which allows more time for clinical decision‐making at the point of care.

The incorporation of the patient portal into clinical practice has shown promise, particularly for chronic health conditions, including asthma [5, 6, 7]. Fiks and colleagues developed a pediatric asthma management tool for parents of children with asthma in general pediatric clinics that used a web‐based portal interface to collect data for patient and physician‐facing care plan decision support [8]. Their studies found improvements in asthma management among parents who used the interface. However, the module was not necessarily widely used by families in practice. This elucidated a further need to focus on the engagement of parent end‐users to enhance the uptake of health management tools such as this.

Motivated by this need, our team has studied patient portals and user experience (UX) design strategies to enhance the likelihood that the portal for asthma management tools will be used in practice [9, 10, 11, 12]. Through this lens, we are developing a portal for asthma care (PAC) management module that collects patient‐entered data through a previsit portal asthma intake questionnaire, automates the integration of the portal questionnaire's answers into the provider's progress note, and includes a summary interface to assist the provider with medical care decisions at the time of the visit.

We report here on the first stage of this PAC module developed for patient caregivers (subsequently referred to as parents) of children with asthma to use prior to their child's pediatric pulmonary clinic visits. The goals of the study were (a) to evaluate the validity of the module to capture asthma data as designed, (b) to measure parent‐reported usability of the PAC module, and (c) to understand parent facilitators and barriers of PAC module use to inform future iterations of the module.

2. Materials and Methods

Our work was conducted in the outpatient pediatric pulmonology specialty clinics at the University of California, Los Angeles (UCLA) from May 2022 to July 2023. There were five pediatric pulmonologists staffing three pediatric pulmonology clinics. Our module prototype was built in our EHR (Epic Systems) which has an integrated patient portal, MyChart.

2.1. The Portal for Asthma Care (PAC) Management Module

The PAC module development was informed by our previous patient portal review and UX design work [9, 10, 11, 12] within the parameters of our EHR. The module is comprised of a previsit intake questionnaire for the parent to complete through the electronic patient portal, a healthcare provider note template that integrates the patient‐entered data into the visit progress note, an asthma order set, a physician‐facing summary of relevant asthma data to help inform clinical decisions, as well as an asthma action plan in the EHR (see Figure S1). During enrollment, usually 2–3 days before their visit, parents were provided instructions about how to access the assigned intake questionnaire. For this prototype stage, the intake questionnaires were manually assigned to the patient's account by our research coordinator. The questionnaire was then accessed by the parents through a dropdown menu in their child's portal. In the future, the intake questionnaires will be assigned to the patient's account automatically as part of the visit pre‐check‐in workflow.

The intake questionnaire is modeled off our current paper‐based intake form and collects guideline‐based, clinically relevant data about the patient's asthma [2, 13]. Impairment, risk, and adherence data are captured by a validated instrument—the Pediatric Asthma Control and Communication Instrument (PACCI) [14]—embedded into the portal intake questionnaire. For new patients, the intake questionnaire content includes questions about the reason for visit, medical history, comorbidities, environmental exposures, adherence, symptoms (impairment and risk), and barriers to care. For return patients, the shorter intake questionnaire focuses on reason for visit, symptoms (impairment and risk), adherence, and barriers to care. See Tables S1 and S2 for the intake questionnaire content. Most questions are framed as closed‐ended response categories with choices displayed with buttons or lists; and include the option of “other,” “none,” or “does not apply.” To reduce burden on end‐users, cascading questions are only displayed if applicable (e.g., if an answer is “yes,” then follow‐up questions will appear to gather more detailed information). There are also limited free‐text answer choices. The answers are stored as discrete data elements in the EHR data warehouse.

Prior to the intervention, the status quo was that healthcare provider note templates were text‐based. The PAC module implements a more advanced technology using Epic's SmartForms. These are templates that feature structured data capture and text‐generation functionality to facilitate data capture and documentation during a visit. Through operational coding of the intake questionnaires, these special forms integrate the portal‐entered data into the provider progress notes using values assigned to SmartData elements (SDEs). Benefits of these forms include that the intake questionnaire answer buttons are visible to the physician in real‐time (e.g., before or during a visit) to edit if responses change after submission and data can be captured for clinical decision support. In addition to the SmartForm note template, relevant asthma data from the intake questionnaires and EHR are also summarized for the physician in a dedicated section within the documentation interface to help with clinical decision support (e.g., a summary of adherence, patient concerns, comorbidities, control/severity, imaging results, influenza immunization date, labs related to asthma phenotyping, risk factors for exacerbations, and triggers).

2.2. Recruitment and Data Collection

Through convenience sampling, we recruited a diverse sample of parents whose children (ages 0–11 years old) had an upcoming clinic visit for asthma via telephone. If participants were not portal users initially, they were provided instructions about how to activate their portal through the MyChart app or web browser. After parent enrollment, the intake questionnaires were assigned and participants were also asked to complete electronic surveys emailed via Research Electronic Data Capture (REDCap) [15] to collect demographics as well as feedback about the portal intake questionnaires.

Feedback about the portal module included quantitative and qualitative methods. We measured usability was through a validated measure pertaining to new technological tools, the System Usability Scale (SUS) [16]. The SUS is comprised of 10 Likert scale (1 = strongly disagree; 5 = strongly agree) questions and includes a scoring system of 5 positive oriented questions such as “I felt very confident using the system” and 5 negatively oriented questions such as “I found the system unnecessarily complex.” Several studies have examined the SUS scoring and through meta‐analyses, an average usability score on the SUS is considered to be ~68 out of 100; and ideal scores are considered to be ~78 (85th percentile) or above [16, 17].

To assess barriers and facilitators, we included three open‐ended questions about the process in the REDCap surveys: “What did you think was easy,” “What did you think was hard,” and “What would you change.” [18]. Participants who completed the previsit intake questionnaire and postmodule surveys received a $25 Amazon gift card. Participants were sent one reminder to complete the intake questionnaire before the visit and up to five reminders to complete the postmodule surveys. This study was approved by the UCLA Institutional Review Board (IRB#20‐000347) and informed consent was obtained per protocol (see Ethics Statement).

2.3. Outcome Measures

We performed a manual chart review of visit progress notes with and without participant use of the PAC module prior to the visit to assess validity of the module to capture data in the providers' progress notes. Within these progress notes, we assessed data capture related to key elements of asthma management. We recorded the presence or absence of whether the note addressed 13 key guideline‐based elements of asthma used to help providers make clinical treatment and management decisions: adherence, asthma action plan, barriers to care, comorbidities, control classification, goals for their asthma, medication, missed school days, relevant labs/imaging, risk (at least one category addressed of the risk criteria from the NIH asthma guidelines), severity classification, symptoms (at least one category from the impairment criteria from the National Institutes of Health's [NIH] asthma guidelines), and triggers.

For return patient parents (n = 40), we compared the PAC intervention note with the most recent clinic visit note by the same provider before the PAC intervention. For new patient parents (n = 5), we reviewed the clinic visit note by the same provider following the PAC intervention. With small exception (n = 2), the clinic visit notes with and without the PAC module were documented by the same physician pre‐ and postintervention.

2.4. Statistical Analysis

We used both descriptive and inferential statistical methods to analyze the data. Descriptive statistics (Table 1) were used to summarize participant demographics, including age, gender, ethnicity, education level, internet access, and portal usage. For functionality metrics, the percentage of key asthma‐related data elements captured in physician notes, we compared data pre‐PAC and post‐PAC using the Wilcoxon signed rank test, as the paired data collected from the same participants were not normally distributed. The results were reported as median and interquartile range (IQR). Usability metrics, specifically the SUS scores, were reported as mean ± standard deviation (SD). We used SAS Version 9.4 (SAS Institute Inc., Cary, NC) for data analysis and considered p value < 0.05 as statistical significance. For our qualitative data, we used the constant comparative method to identify the range and commonality of responses the three open‐ended questions about barriers, facilitators, and ways to improve the module [19].

Table 1.

Descriptive statistics of demographics and characteristics of parents.

Demographics and characteristics of parents	N = 44
Age
18–24	1 (2%)
25–29	1 (2%)
30–39	13 (30%)
40–49	28 (64%)
50–59	1 (2%)
Gender
Female	37 (84%)
Male	7 (16%)
Nonbinary	0 (0%)
Other/prefer not to answer	0 (0%)
Race
Asian	5 (11%)
Black	1 (2%)
White	28 (64%)
Multiple races	3 (7%)
Other/prefer not to answer a	7 (16%)
Ethnicity
Hispanic	12 (27%)
Non‐Hispanic	28 (64%)
Prefer not to answer	4 (9%)
Insurance
Employer	38 (86%)
Government or government subsidized	5 (11%)
None	0 (0%)
Other/prefer not to answer	1 (2%)
Highest education completed
< High school	0 (0%)
High school	3 (7%)
College/some college	20 (45%)
Graduate school	19 (43%)
Other/prefer not to answer	2 (5%)
Who cares for child with you
Other parent/partner	27 (61%)
Family/friend	2 (5%)
No other help	15 (34%)
Other/prefer not to answer	0 (0%)
How many years used portal
< 1	4 (9%)
1–4	19 (43%)
5–10	20 (45%)
> 10	1 (2%)
Prefer not to answer	0 (0%)
Stable internet at home
Yes	44 (100%)
No	0 (0%)
Preferred method to log into portal (multiple answers)
Phone app	32 (73%)
Tablet app	1 (2%)
Desktop/laptop app	5 (11%)
Desktop/laptop online	21 (48%)
Prefer not to answer	0 (0%)

^a Other race = Alaska Native, Native American, Native Hawaiian, other Pacific Islander.

3. Results

In the following section, we report social demographics, how much key asthma care data elements were captured by the compared with the status quo, and the usability of the PAC through quantitative and qualitative measures.

3.1. Demographics

Forty‐five parents used the PAC intake questionnaire prior to their pediatric pulmonology visit and 44 completed the demographic data survey. Of those who completed the survey, 84% were female and 36% were Hispanic/Latinx. In addition, 88% attended/completed college and/or graduate school, 86% had insurance through their employer, and 100% had access to a stable internet connection. The most preferred method to access the portal was through a smartphone application (73%). Most of the participants either had 1–4 years (43%) or 5–10 years (45%) of experience using a patient portal. Further details are shown in Table 1.

3.2. Data Captured

The PAC increased the percent of the 13 key asthma data elements documented in physician clinic visit notes. Without the PAC intervention, 77% (IQR 69%–77%) of the key elements were captured and with the PAC, the percent of relevant asthma data captured was 92% (IQR 92%–100%) (p < 0.001; Figure 1). The most increased documentation was noted for key elements of asthma goals, missed school days, and barriers to care.

Figure 1.

Percent of the 13 asthma data elements captured by healthcare providers in their clinic visit notes without and with the portal for asthma care (PAC).

3.3. Usability

First, we analyzed SUS scores and found that the average score was 83.8 ± 11.2 (out of maximum 100). This is considered a high usability score, given that an average SUS score is considered near 70 [16, 17].

Second, we analyzed our qualitative questions, “What was easy?,” “What was hard?,” and “What would you change?” Overall, the feedback was positive. Themes related to aspects that facilitated use of the tool were time savings, ease of use, and technological advantage. Most reported that the process was more convenient than paper forms because it saved time, allowing them to complete the forms at their own convenience and on the go. They appreciated having this option especially if the appointment was virtual. Parents also reported that the PAC made it easier once they arrived at the clinic visit for reasons such as “At the appointment I have a child with me, which makes (paper intake forms) more difficult to do.”

Themes related to potential barriers of use were access, clarity of questions, and technological disadvantages. Some reported difficulty remembering their password and remembering to complete the intake questionnaires before the visit. One confirmed that a reminder 1–2 days before the visit would typically be helpful. There was feedback that some of the answer options were not necessarily applicable to their child. One noted a question was displayed too wide for the phone, so they needed to turn their phone to landscape orientation. Some participants reported the intake questionnaire was difficult to find through the menu options. Of note, this process of finding the questionnaire in the portal was due to the prototype nature of this stage of being assigned manually; however, this issue should resolve once the tool is integrated into routine care and assigned automatically as part of the visit check‐in process. See Table S3 for detailed responses.

4. Discussion

Our study aimed to evaluate (a) the functionality of the PAC module through key asthma elements captured, (b) the usability of the PAC module through SUS scores, and (c) to understand facilitators and barriers of the PAC module prototype to inform future iterations of the module. At this stage of development, we found the PAC module was able to capture significantly more key asthma elements in the provider notes and received high usability scores from parents, indicating that this is a potentially useful modality to address care needs as well as streamline patient visits for asthma. In addition, parental feedback indicated that using the portal to capture patient‐reported asthma prior to visits was convenient and preferred over the status quo, indicating promise that the PAC could be used in routinely in practice.

Our module highlights the incorporation of an asthma management module that includes an EHR‐integrated patient portal into care, which has several advantages for parents over other standalone technological interventions (e.g., mobile apps or separate web‐based portals) [5]. These include more involvement in their child's asthma care because the portal is directly linked to their medical records (the EHR) and provides functionalities directly related to management such as provider messaging, educational material access, refill requests, etc. In addition, healthcare providers can use the patient portal to tailor information to patient needs in the after‐visit instructions, communicate with families directly through messaging, and provide education about asthma management.

While portals provide many valuable functions and have been effective when incorporated into management, like any electronic‐based intervention, there is a known gap between creating an electronic tool and its actual uptake in practice [8, 20, 21]. One reason for this is because end‐users are being asked to perform a new task, which can be potentially burdensome. Users need to learn about the new tool, understand how to use it in practice, and recognize its purpose and value. To address this challenge, we have been incorporating UX design approaches into our development of electronic health management tools [9, 10, 11, 12]. These approaches capture end‐user feedback about tool usability and considers real‐time input to revise the system accordingly. This can bolster the uptake of electronic tools such as PAC, as well as improve tool functionality through design refinement [22, 23, 24]. The PAC module has potential to empower patients more in their healthcare by partnering with physicians to provide information before the visit, resulting in more time to address their concerns and needs surrounding their asthma. It also increases their exposure to the patient portal if they are not familiar with it as they learn about its functionalities during the enrollment process.

While the PAC appears promising for parents to use in practice, our study has limitations. This work was executed in a controlled research setting where each intake questionnaire and follow‐up survey were manually assigned by our research assistants and may not reflect results in typical use. Our sample size was fairly small, but the feedback appeared to reach thematic saturation and we were able to deploy actionable steps for future iteration. Also, the participants who enrolled were likely different than those who did not, which could influence their feedback about our PAC module. We only enrolled English‐speaking/reading participants; and participants fairly well‐educated (proxy for socioeconomic status), so work such as this may not extrapolate to all populations and/or neglect specific demographics. Also, we did not perform analysis related to patient age to determine if there were differences related to parental usability or module feedback. In addition, all data captured in the module is self‐reported by the parent and as children age, they spend less time with parents so data collected may not be representative of the true clinical picture. Finally, some suggestions for optimization by participants (e.g., content of asthma questionnaires or logistics of accessing the questionnaire) are not always completely modifiable due to the standard of clinical care guidelines or technical limitations of the EHR.

For the next iteration of our PAC module, we will incorporate as much of the parent feedback as possible and optimize the module through this iterative approach. We are also incorporating end‐user feedback of the health providers into the design and implementation process. In addition, we are evaluating this module in the adolescent population and will continue expand the module's use in practice.

Author Contributions

M. K. Ross: conceptualization, investigation, funding acquisition, writing – original draft, methodology, validation, writing – review and editing, project administration, formal analysis, data curation, supervision, resources. E. J. Clark: investigation, writing – review and editing, formal analysis, project administration, data curation. W. Chan: writing – review and editing, investigation, formal analysis. D. Kafashzadeh: writing – review and editing, data curation, formal analysis. I. Radparvar: writing – review and editing, data curation, investigation, formal analysis. E. Gao: writing – review and editing, investigation, formal analysis. A. Gomez: software, writing – review and editing. M. Tran: writing – review and editing, software. M. S. Sim: formal analysis, writing – review and editing, resources. G. Rong: writing – review and editing, formal analysis. S. Friedman: writing – review and editing, investigation, methodology, project administration. P. G. Szilagyi: writing – review and editing, methodology. G. Ryan: writing – review and editing, methodology. A. A. T. Bui: writing – review and editing, methodology.

Ethics Statement

The study protocol was reviewed and approved by the UCLA Institutional Review Board (IRB) Approval #20‐000347. Our team recruited patients primarily through telephone calls and assessed comprehension through a study information sheet and the consent document.

Consent

Patients confirmed their verbal consent with our research assistant through an electronically signed consent.

Conflicts of Interest

The authors declare no conflicts of interest.

Supporting information

Supporting information.

Data Availability Statement

The data that support the findings of this study are available from the corresponding author upon reasonable request.