Acceptability and Feasibility of a Mobile Web Application to Help Identify Nontraumatic Abdominal Patient-Reported Symptoms in the Emergency Department: A Pilot Study

Abstract

Objectives

We developed a mobile web application to help identify nontraumatic abdominal patient-reported symptoms (PRS) in patients who visited the emergency department (ED). Patients can use this during waiting times to facilitate patient history-taking by the physician. The study aimed to evaluate the feasibility and acceptability of PRS usage in the ED by patients.

Methods

A mixed-methods study was conducted at the ED of an academic tertiary hospital in Seoul, Korea. Adult patients aged ≤65 years presenting with abdominal symptoms were enrolled. We assessed PRS completion, time, and help required. Patient surveys and semi-structured interviews with patients and physicians were used to evaluate feasibility and acceptability.

Results

A total of 30 patient surveys were analyzed, and in-depth interviews were conducted with 6 patients and 6 doctors. All participants completed the PRS in an average of 7.6 (SD, 5.0) minutes. Sixteen patients (53.3%) needed help with content, and 15 (50.0%) required technical assistance. The PRS was rated as acceptable based on the diffusion of innovation theory, with an average of 3.9 (SD, 0.76) out of 5. Patients reported that the PRS helped them communicate symptoms clearly but raised concerns about its usability for older patients or those in pain.

Conclusion

In our pilot study, the PRS was feasible to use but required improvements in usability. Patients were acceptable to PRS use in the ED.

The Bottom Line.

This study evaluated the feasibility and acceptability of a mobile web application designed to help patients organize abdominal symptoms before history-taking in the emergency department. In a pilot study of 30 adult patients at a tertiary hospital in Seoul, Korea, all participants completed the symptom-reporting process within an average of 7.6 minutes. Most patients and physicians found that the application improved symptom communication and medical assessment efficiency. Therefore, the patient-reported symptom tool has the potential to serve as an effective supplementary instrument for history taking and improving patient-physician communication in emergency care.

1. Introduction

1.1. Background

Despite the significant role of communication in shaping patient experience, satisfaction, and outcomes in the emergency department (ED),1, 2, 3 effective communication remains challenging.^4,5 Moreover, history-taking – the essential first step in diagnosis and treatment⁶—is often disregarded because of the focus on the promptness of medical service,³ and the frequent lack of a preexisting doctor-patient relationship may further complicate communication.⁷

1.2. Importance

Although the significance in medical history-taking, previous studies have shown that this critical process is fraught with challenges: (1) patients are inconsistent in remembering and communicating symptoms^8,9; (2) face-to-face communication may be inaccurate due to anxiety faced by the patient^6,10; (3) physician bias toward the patient may affect the history-taking process¹¹; and (4) language barriers limit communication.¹² Moreover, the experience of the physician may influence the performance of history-taking in time-pressured situations and may result in the omission of critical information.¹³ Therefore, the traditional heuristic approach to medical history-taking may be, at times, inconsistent and prone to error.

Technological solutions present opportunities to improve medical history-taking. These solutions can help patients organize their problems during waiting times and potentially enhance the efficiency of clinical encounters. However, questions remain regarding the patient feasibility and acceptability of such tools, especially the impact on patient flow, patient-perceived privacy concerns, and varied technological literacy among patients.

1.3. Goals of This Investigation

This study aimed to investigate the patient experience of using a mobile patient-reported abdominal symptom system, with a focus on operational feasibility and acceptability. We focused on adult patients presenting with abdominal symptoms in the ED, given that abdominal pain is one of the most common reasons for emergency visits.

2. Methods

2.1. Study Design and Setting

This study used a mixed-methods design, incorporating both quantitative surveys and qualitative interviews to evaluate the feasibility and acceptability of the patient-reported symptom (PRS) system.

This study was conducted in the ED of an academic tertiary hospital in Seoul, South Korea, between March 2 and March 14, 2022 (on weekdays from 9 am to 6 pm). There is an average of 78,000 ED cases per year. The study protocol was approved by the Institutional Review Board (IRB) of the study site (IRB No. 2022-02-009-002). We obtained informed written consent before enrolling the participants by following the IRB policy.

2.2. Selection of Participants

Given the exploratory nature of the pilot study, which was conducted without hypothesis testing, we selected 30 participants to collect both quantitative and qualitative data. This sample size was chosen to facilitate both feasibility assessments and the acquisition of valuable qualitative insights, providing a manageable yet informative dataset for an initial pilot study. We included adult patients under 65 years old with nontraumatic abdominal complaints, as younger adults are generally more proficient with mobile technology, making them ideal for evaluating our prototype’s usability and effectiveness.^14,15 Patients with high-severity Korean Triage and Acuity Scores (KTAS) (1-2), visual or motor impairments, difficulty using mobile devices, or suspected COVID-19 or airborne diseases were excluded to minimize confounding factors and ensure safety.

2.3. Study Process

After triage, patients in the waiting room were screened for consecutive enrollment based on inclusion and exclusion criteria. Patients who provided written informed consent to participate in this study were enrolled. The enrolled participants used the PRS system on their mobile devices to fill out the items required to report their symptoms and capture the last report page, which organized their symptoms based on their report. The physicians then read the report and acknowledged the patient-reported information before history-taking. As there was an additional waiting period after triage and vital sign measurements, patients had sufficient time to complete PRS before meeting the physician. After completing the history-taking process, all participants were required to complete the survey (Fig S1).

2.4. PRS Report Development

PRS was a prototype mobile web service developed by a board-certified emergency medicine physician using a patient-reported outcome development process¹⁶ (Fig S2).

We developed a prototype web-based PRS data-entry questionnaire tree to assess patients who came to the ED. We targeted chief complaint categories for nontraumatic abdomen-related symptoms because of the frequency of these symptoms and the depth of information needed for abdominal symptoms.

The PRS system comprises 2 parts. The first is a PRS data-entry section, and the second is the patient report (Fig S3). The former collected basic demographic information and details on abdomen-related symptoms. Subsequently, a review of systems, past medical history including medication and allergy history, and any questions or requirements patients had for physicians were asked. We referred to the Objective Structured Clinical Examination for abdominal pain assessment items and the essential questions for the differential diagnosis of abdominal pain.^17,18 Most questions were asked to collect structured information with multiple-choice questions and limited text input. After inputting all the required information, a patient report was generated by reorganizing and transforming the patient input data into a clinical report format.

2.5. Outcomes

The primary outcome of our study was the operational feasibility of PRS usage in the ED. As an early prototype assessment, we applied the Technical, Economic, Legal, Operational, and Scheduling feasibility framework,¹⁹ evaluating completion rates, ease of use, and help required when using the PRS data entry.

To assess acceptability, we applied a refined version of Rogers' diffusion of innovations (DOI) framework through the survey. The DOI framework is a widely used model for information technology adoption in health care.19, 20, 21, 22 This framework evaluates 5 attributes—relative advantage, compatibility, complexity, trialability, and observability—and was used to assess the acceptability of the PRS in the ED and users' readiness to adopt new technologies.^23,24 We constructed questionnaire items on a Likert scale regarding the acceptability of the PRS, referring to prior studies that used the same framework to evaluate the acceptability of a new type of service in health care.^20,21

2.6. Quantitative Survey

2.6.1. Data collection and measurements

Patient demographics, including age, sex, ED arrival type, KTAS level, chief complaint, and ED disposition, were prospectively collected from electronic health records. In this study, the term “sex” refers to biological differences between males and females. While participants used the PRS in the waiting room, researchers recorded completion status, time taken, and any assistance needed for content or technical issues.

After using the PRS, patients completed a survey based on the DOI framework. The survey consisted of 4 parts: (1) demographics and confidence in mobile device usage; (2) acceptability based on the DOI theory; (3) content of the PRS data entry; and (4) open-ended questions about the general experience and expected merits, potential problems, and areas of improvement in using the PRS. A 5-point Likert agreement scale was used to rate the responses (Supplementary Appendix 2). The survey was conducted online via a link sent through a text message.

2.6.2. Data analysis

Patient characteristics were described using descriptive statistics. Categorical variables were expressed as frequencies and percentages, whereas continuous variables were expressed as means and SDs. For questions asked with negative wording, the Likert scale was reverse-coded. We used a mixed-methods approach as the open-ended survey items were examined using the DOI framework to better illustrate the quantitative responses.

2.7. Qualitative Interview

2.7.1. Interview participants

We conducted semistructured interviews to assess the acceptability and practical challenges of using the PRS. Two groups participated: (1) a subset of patient participants who had used the PRS during their ED visit and (2) ED physicians who had interacted with those patients and their PRS reports. A total of 12 interviews (6 patients and 6 physicians) were conducted, considering data saturation and resource constraints. Based on Nielsen and Landauer’s²⁵ research, 5 participants can identify about 85% of interface issues. We aimed for at least 5 participants and stopped due to physicians’ availability and data saturation. For a balanced perspective, we matched the number of interviewees of doctors and patients.

2.7.2. Interview process

We conducted all interviews in person, using a consultation room within the ED, to minimize bias and disruptions in data collection. Physicians were interviewed outside their duty hours, while patients participated after completing their ED visit with consent. A semistructured template covering general experience, pros and cons, potential issues, and improvement suggestions guided the discussions (Supplementary Appendices 2 and 3). All interviews were audio-recorded and transcribed verbatim.

2.7.3. Data analysis

Two researchers independently reviewed each transcript, applying an inductive coding approach to capture emerging themes. Through iterative discussion, these codes were refined and grouped into 3 overarching categories: (1) positive responses, (2) negative responses or concerns, and (3) areas of improvement. We further stratified these themes using the physician and patient perspectives to highlight any differences. Discrepancies in coding were resolved by consensus. Representative quotations and summaries of these themes were later presented in the Results section.

2.8. Patient and Public Involvement

None.

3. Results

A total of 39 patients were consecutively screened for the study, and among them, 5 declined to participate, citing pain or weakness as their reason. Four patients completed the PRS but did not submit the online survey. Finally, 30 patients who completed the survey were included in the analysis. An in-depth interview was conducted with 6 patients and 6 doctors who evaluated the patients using the PRS (Fig 1). The baseline characteristics of the study participants are presented in Table 1.

Figure 1.

Study flow. ED, emergency department. CYREN, Communicate Your Report through our Emergency Network.

Table 1.

Baseline characteristics of participants.

Clinical variables	N = 30, n (%)
Age (y), mean (SD)	46.9 (13.4)
19-39	9 (30.0)
40-59	11 (36.7)
>60	10 (33.3)
Sex
Male	11 (36.7)
Female	19 (63.3)
User
Patient	27 (90.0)
Family member (spouse)	2 (6.6)
Family member (daughter)	1 (3.3)
ED arrival type
Walk in	25 (83.3)
EMS service	3 (10.0)
Private transportation	2 (6.6)
KTAS
1 (most urgent)	0 (0)
2	1 (3.3)
3	16 (53.3)
4	13 (43.3)
5 (least urgent)	0 (0)
Chief complaint
Abdominal pain	18 (60.0)
Abdominal distention or discomfort	3 (10.0)
Flank pain	4 (13.3)
Nausea	2 (6.6)
Melena	1 (3.3)
Poor oral intake	1 (3.3)
Jaundice	1 (3.3)
ED disposition
Admission	10 (33.3)
Discharge	20 (66.6)
Confidence in using a mobile device
Excellent	12 (40.0)
Good	7 (23.3)
Fair	7 (23.3)
Poor	2 (6.7)
Very poor	2 (6.7)

ED, emergency department; EMS, emergency medical system; KTAS, Korean Triage and Acuity Score.

3.1. Feasibility of Using PRS

All patients were able to complete the PRS data entry. The average time taken to complete the data-entry part was an average of 7.6 (SD, 5.0) minutes (Table 2). The longest time was 20 minutes, and the shortest was 2 minutes. The distribution of time for completion, age, and events of help requested by participants is shown in Figure S4.

Table 2.

Feasibility of the mobile patient-reported symptom system.

Variables	N = 30, n (%)
Completion of PRS	30 (100)
Time (min) to finish, mean (SD)	7.6 (5.0)
Required help
Content related	16 (53.3)
Describing the onset of symptoms	9 (30.0)
Describing symptom characteristics	6 (20.0)
Describing symptom severity	1 (3.3)
User interface related	15 (50.0)
Difficulty filling out the time information	4 (13.3)
Selecting answers based on multiple-choice questions	3 (10.0)
Discomfort with the text’s small font size	2 (6.7)
Failure to find free-text fields	3 (10.0)
Failure to go on to the next page	3 (10.0)

PRS, patient-reported symptoms.

A total of 16 participants (53.3%) required help with content-related issues, and 15 participants (50.0%) required help with technical issues, such as those related to the UI (Table 2). The most common content-related requirement was related to describing the onset of symptoms. Some participants required help in describing their pain, especially when it changed over time. Help required owing to technical issues was related to the user experience and UI design in multiple-choice questions, failure to go on to the next page, or selection of free-text fields.

3.2. Survey Outcomes

PRS’ acceptability based on the DOI theory was good, with a total mean score of 3.9 ± 0.76 (mean ± SD) out of 5. The results are shown in Figure 2.

Figure 2.

Survey outcomes regarding diffusion of innovation and content of PRS. ED, emergency department; HCP, health care provider.

3.2.1. Relative advantage

Most items showed positive expectations compared with the status quo. Expectations for better health care efficiency (4.2 ± 0.58), providing better communication with physicians (4.2 ± 0.66), easier usage of the ED (4.2 ± 0.66), and faster conveyance of health information to physicians (4.2 ± 0.66) were especially high.

In the open-ended questions, 5 patients stated that the PRS had a relative advantage, as it could organize their problems in detail. Three participants commented that the PRS was helpful because they could communicate their symptoms before seeing the doctor, and 2 stated that they could communicate better when it was difficult for them to talk due to discomfort or pain (Table 3).

Table 3.

Interview responses by physicians and patients.

Responses	Physicians	Patients
Positive responses	Well-organized symptoms The chronological order of problems was helpful Helpful in gathering detailed information Reduces history-taking time	Helps organize and remember symptoms Conveys symptoms clearly and easily Expectations of faster care
Negative responses	Problems with the report’s credibility Mismatch in the meaning of symptoms between doctors and patients Concerns about bias and misleading information Too much information for complex patients	Might be problematic for the elderly or patients in pain Data entry was too limiting to express problems

3.2.2. Compatibility

Regarding responses to questions on compatibility with the health care process, participants considered the PRS useful in the ED (3.8 ± 0.46) and agreed that it was compatible with the expected health care service in the ED (3.8 ± 0.57) and that it improved the health care flow process (3.8 ± 0.65) (Fig 2).

Regarding the open-ended questions on compatibility, 7 patients said they expected to receive time-efficient and fast medical communication and services with the PRS. However, one participant commented that the doctor asked the same questions regarding the participant’s symptoms, even though they had reported them through the PRS (Table 3).

3.2.3. Complexity

In response to questions on complexity, the PRS was not considered to make health care in the ED more complex (3.5 ± 0.73) and was not considered difficult to use due to physical pain or discomfort. However, it had a low average score (2.9 ± 0.97) (Fig 2).

Regarding the open-ended questions, 7 participants were concerned that other elderly patients and patients unable to use their mobile devices would have difficulty using the PRS. Furthermore, 6 patients were worried that it would be difficult to use when in pain or severely ill (Table 3).

3.2.4. Trialability and observability

Participants said that they would use the PRS in the future (3.9 ± 0.64) and would recommend it to other patients (3.8 ± 0.63).

3.2.5. Content

In the structured survey, for questions regarding the content, participants replied that the general words (3.9 ± 0.52) and medical terms (3.8 ± 0.53) were clear, and the contents of the PRS were easy to understand (3.8 ± 0.66) (Fig 2).

Regarding the open-ended questions about the content, 4 participants said the questions were obscure and needed to be more specific and clearer. Three participants replied that it was difficult to express their symptoms in this structured data-entry format and suggested that free text would be better for expressing the problem rather than limiting the expression to given answers (Table S1).

3.2.6. Semistructured interview

We reviewed the transcribed results of the interviews and examined the positive and negative responses as well as concerns or areas of improvement from physicians and patients. The results are summarized in Table 3. Additionally, we present the physicians’ and patients’ dictations in Table S2.

3.2.7. Positive responses

3.2.7.1. Physician

Almost all physicians responded that the patients’ reports were well-organized and concisely detailed the patients’ complaints, allowing them to quickly assess and comprehend the condition prior to history-taking. Additionally, physicians expressed satisfaction with the thorough review of the specific contents. They responded that the report was also helpful because the symptom information was chronologically ordered.

3.2.7.2. Patients

The most favorable perception of patients was that they could communicate their symptoms clearly and easily. Patients believed that the PRS was beneficial for summarizing and reminding them of their symptoms. Moreover, the participants stated that they felt well taken care of and more at ease. They believed it would help them receive medical care more quickly.

3.2.8. Negative responses or concerns

3.2.8.1. Physicians

Although they were satisfied with the patient reports, the physicians expressed concerns about their reliability. A reason for this may be a mismatch between what doctors and patients perceive as the primary problem or different meanings that both might give to the same symptom term. This concern about the credibility of the reports leads to concerns about bias regarding patient problems. Moreover, several physicians were concerned about excessive information in the reports on complex patients, which may result in longer consulting times.

3.2.8.2. Patients

Although the patients were able to complete the PRS data entry, some felt limited by the multiple-choice questions or felt that the data entry was too limiting to express a complex problem.

4. Limitations

Our study has some limitations. Its descriptive nature, small sample size, and single-center design may limit generalizability. Additionally, the exclusion of high-severity KTAS patients, those over 65, and individuals with language barriers may affect applicability. These factors should be considered when interpreting the results. Lastly, the study focused solely on abdominal symptoms. Future research should expand symptom coverage, improve usability, and integrate the PRS into real-world workflows, such as electronic medical record (EMR) integration, to assess its impact on ED efficiency.

5. Discussion

In our study, the ability to clearly communicate symptoms through the PRS was a common advantage, as described in the interviews with patients and physicians. The patients reported being able to organize their symptoms before meeting the doctor and felt they were receiving better patient-centered care. Our findings align with a previous study on a tablet-based ED self-assessment system, where patients felt it improved doctor communication and care quality.⁶ Using this PRS system can allow patients to prepare and better advocate for themselves, and physicians can review patient information as a starting point before asking questions.

Our findings provide insight into the real-world disparities observed in using patient-reported information for computer-generated medical reports. A previous study found that although computer-generated medical reports from patient-reported data were higher in quality²⁶ than physician-generated ones, they did not improve patient satisfaction²⁷ or reduce physician workload.²⁸ These may be explained by physicians' concerns about report reliability, potential bias, and excessive information, particularly for complex patients, which could add to their workload. Although PRS aids in information gathering, discrepancies between patient-reported and physician-recognized problems have been noted.²⁹ Physician interviews have also revealed differences in symptom perception. Therefore, PRS cannot be a substitute for medical history-taking but should be limited to a supportive role.

Some patients expressed concerns that those in pain or the elderly might have difficulties with the PRS, and our study found that older patients needed more time and assistance for symptom entry. Prior studies indicated that factors such as pain level, age, and digital literacy affect the ability to complete these reports.30, 31, 32 Therefore, these factors should be considered to identify which patient groups would benefit most from PRS in the ED. Although language barriers were not assessed—because the application was tested in an environment with minimal language issues—future multilingual support could help mitigate such barriers.

This study provided key insights to enhance PRS and improve its real-world applicability. Patients frequently required assistance with content-related issues, particularly in describing their symptoms, and found the structured data entry too restrictive. To address this, incorporating unstructured text inputs and large language models may enable more comprehensive symptom descriptions. Additionally, user interface (UI)-related difficulties highlight the need for better user experience and interface design to enhance usability.

From a technological and workflow perspective, privacy and security concerns related to patient data usage should be addressed. The fast-paced nature of the ED and limited staff availability may also pose challenges in adopting and effectively utilizing PRS. Future research should expand symptom coverage, improve diagnostic evaluation, and explore strategies for seamless EMR integration to enhance real-world applicability and assess the impact on ED efficiency.

In conclusion, our mixed-methods study demonstrated the acceptability and feasibility of PRS in helping patients organize their symptoms before meeting with physicians. However, it also highlighted the need for further enhancements to facilitate more effective usage.

Author Contributions

MKC and WCC conceived the study and designed the trial. SY collected the data. SY and SH provided statistical advice on study design and analyzed the data. MKC and WCC inspected the analyzed data. MKC, SH, SY, and WCC drafted the manuscript, and all authors contributed substantially to its revision. WCC takes responsibility for the paper as a whole.

Funding and Support

This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. RS-2022-NR070548).

Conflict of Interest

At the time the study was designed and performed (March 2022), analyzed, and the manuscript was written, MKC had no conflicts of interest. After completion of the study, the results of the study inspired MKC to found CyrenCare, Inc, later in January 2023.

SH, SY, and WCC report no conflicts of interest.