Development and usability evaluation of a mHealth application for prehospital triage

Abstract

This study developed and evaluated a mobile health (mHealth) application for prehospital triage. The effective triage of undifferentiated patients is crucial for appropriate prehospital care and mHealth applications offer potential support by classifying patient conditions prior to hospital arrival. This two-stage applied study, conducted in 2023, began with determining functional requirements through a literature review and a questionnaire-based survey, informed by existing guidelines. A conceptual model was then developed using Microsoft Visio 2021. The application then was developed using Android Studio in Java and Kotlin languages. In the second stage, the application’s usability was evaluated by involving 32 paramedics and EMT technicians through using the Mobile Application Usability Questionnaire (MAUQ). Key application features included comprehensive information forms, triage level calculation based on the Revised Trauma Score (RTS), hospital navigation, and structured case report generation. Ease of use received the highest score (4.35 ± 0.47), while interface and satisfaction obtained the lowest scored (3.84 ± 0.79). Overall user satisfaction was high (4.12 ± 0.59; Max: 5.00). Results suggest that this mHealth application can effectively support prehospital care by addressing user needs and providing essential functionalities.

Supplementary Information

The online version contains supplementary material available at 10.1038/s41598-025-06952-4.

Background

Triage refers to a dynamic decision-making process which prioritizes patients and their needs for medical interventions^1,2. Medical triage can be classified into two main categories, including: hospital triage and prehospital triage. Today, what affects human life and health the most are medical emergencies following chronic health conditions, accidents, natural, or man-made disasters. In times of emergencies when demand exceeds resources, allocating the most available resources to those who need them more than others and saving as many lives as possible can be a game-changing process, which results in a significant reduction in mortality and morbidity following efficient prehospital care^3–5. In prehospital settings, paramedics would be able to provide patients and injured who are in more critical condition with care and the utmost treatment possible, considering resource limitations, including time, human force, and equipment^3,6.

The prehospital healthcare process commences at the time of the incident and continues up to the patient’s arrival at healthcare facilities. So, immediate, efficient, and structured prehospital care can considerably reduce mortality and disability rates⁵. Accordingly, reports show that about 24 million citizens in Low- and Middle-Income Countries (LMICs) face potential death due to inefficient and inadequate prehospital cares annually. These numbers signify the importance and necessity of a systematic, well-structured prehospital triage process⁷. Not to mention that the prehospital triage is now recognized as a key factor in an efficient emergency medical management⁸.

Over-triage and under-triage are two of many problems associated with poor triage. For paramedics and other emergency responders, not being able to collect, process, or measure data related to the patient and the accident, strongly affects the ultimate decision, which would be the suitable triage level and consequently, interventions following initial diagnosis. In better words, if a patient is tagged with the wrong triage level indicating an undesirable urgency level, they may receive care less or more than needed, and that is when valuable resources are being tragically wasted. In addition, when providers present at a Mass Casualty Incident (MCI) scene expecting a number of casualties, the scarcity of resources becomes a major, life-threatening problem and puts many lives at stake or susceptible to irreversible health conditions, and of course, unnecessary longer hospitalization if being transported to hospital on time^9,10.

Owing to the fact that by providing patients with proper, timely care, especially under emergency circumstances, their chance of survival and health condition would improve, administering triage properly and determining the apt triage level would be even more favorable and crucial than a potential final diagnosis affirmed by in-hospital physicians¹¹. Therefore, one way to minimize time assigned for performing emergency critical measures and care, yet improve the quality aspect, is to employ cutting-edge mobile technologies.

Over recent years, the Internet and mobile technologies have notably altered people’s lifestyle as well as workflows. Human resources working in health and medical domains tend to use mobile devices, including PDAs, smartphones and tablets, more than ever to accomplish their tasks^12,13. “mHealth”, which is a subset of the umbrella term “eHealth”, refers to the use of Information and Communication Technology (ICT), and related devices, e.g., smartphones, in order to collect, transfer, and access health information and services^12,13. The term “mHealth” has been widely utilized in all domains, including education, management, monitoring, diagnosis, and treatment^14,15. In 2019, WHO simply defined mHealth as “The use of mobile and wireless technologies to support health objectives”¹⁶ One of the most prominent achievements in mHealth is the invention of mobile health applications run on smartphones¹⁷.

One practical and effective solution to manage the aftermath and limit damages following an incident or an emergency health condition is to provide victims with timely, adequate and high-quality care, including prehospital triage. Hence, in order to optimize the initial attempts and clinical measures, mobile applications can be widely used by first responders at the scene and during transport time^18,19. Collecting patient information and clinical data on a real-time basis²⁰, determining the appropriate triage level using algorithms^21,22, and the ability to select body regions indicating trauma types²³ are some of the many functions that a prehospital triage mobile application can execute. Currently, there are numerous mobile applications for prehospital care which are primarily designed to triage specific cases such as cardiac or burn emergencies^24–27. To enhance patient care, support diagnostic and documentation processes, various applications are equipped with tools to collect and organize data associated with dispatch calls^28,29. Time plays a crucial role in prehospital settings, where patients need to be treated and/or transferred promptly to prevent further injuries or life-long chronic disabilities. Therefore, there are applications developed to find the optimum route for patient transfer.

Considering the importance of an accurate prehospital care and triage, as well as the wide range of services offered by aforementioned applications and their impact on patients’ clinical status as well as paramedics’ performance, it is essential to design mobile applications, supporting first responders in prehospital settings. As paramedics and EMT technicians are the primary users of such applications, it is necessary for developers to take their needs into consideration. Despite the significance of prehospital care and how a simple data or intervention can alter patient’s life quality or affects paramedics’ performance, studies addressing the use of mobile applications in prehospital setting are limited. In addition, comprehensive applications for prehospital setting are yet limited or with poor functions, which reflects the meagre participation of experts and users in the development of such applications, which can be a consequence of restricted organizational policy and strategies as well as lack of investors. On the surface, a great majority of applications in prehospital care domain at the time of conducting this research were developed mainly for educational purposes and few of them aimed to assist paramedics only with especial cases like stroke or burn emergencies^24–27. Therefore, the present study was conducted to develop and evaluate a mobile application for prehospital triage with a focus on all medical cases including trauma specific cases, burn and cardiac emergencies. The informational forms provided in the application support almost all medical emergency cases with detail description.

Materials and methods

The current applied-developmental study was carried out in two stages in 2023 described below. Relevant articles, guidelines and works were studied in order to gain a bright insight into different aspects of the study including: application’s requirements from technical to informational content, gaps needed to be addressed, prehospital medical care and triage literature. All information gathered through this stage was then applied for application content and interface design.

Application development

The study was commenced with a comprehensive and systematic search on websites including PubMed, IEEE, Web of Science, and Google scholar, using MeSH terms as follows: mobile application, portable electronic application, prehospital emergency care, medical emergency service and triage. The search results were limited mainly to those published between January 2017 and December 2024. The primary selection approach was to consider articles’ titles, key words and abstracts and in what extent they were relevant to the designated topic. After investigating about 400 articles, those discussed mobile applications or systems in emergency, prehospital care and triage were selected for further review. Then, a detailed study was conducted to address key functional requirements, data elements, educational and information content of such applications, and how to determine these requirements in practice to understand the key functions of the mobile application. Features extracted then were reviewed by a group of medical informatics experts and validated by prehospital emergency care experts. To determine the required functions of the application, a research-made questionnaire was developed and evaluated by the expert panel. Also, the face and content validity of the questionnaire was confirmed by the same panel. The questionnaire was composed of 23 questions divided into three categories including user’s demographics, the educational contents of the application, and key functions of the application. The answer for each question was either " it is necessary” or “it is not necessary” or “it is necessary yet not useful”. The Content Validity Ratio was calculated (CVR = 0.6) for each question. Then, 13 paramedics known as end users completed the questionnaire. Finally, based on CVR and experts’ opinion, the ultimate requirements were identified and considered for application design.

Following the application development process, expert panel discussed the content of the application and its main services upon which the conceptual model composed of functional, structural, and behavioral, and models was designed using Microsoft Visio 2021. To translate the conceptual modeling into application development and function designing, Model-Drivel Development (MDD) using Unified Modeling Language (UML) as well as wireframe and UI prototypes were applied. Not to mention all models including the conceptual one was verified by the expert panel. Regarding the programming phase of the study, the application was programmed in Java and Kotlin languages using Android Studio, and tested and debugged by experts to identify and resolve any error before being used in real environment afterwards.

The prototypes were checked by medical informaticians and prehospital emergency care staff to confirm the usability features, and their feedback influenced the design and iteratively refine application’s structure ensuring that the application developed in response to user needs.

Application evaluation

The initial purpose of this study was to evaluate the usability and user perception of the application. As such, the focus was on subjective usability metrics rather than clinical outcome indicators. Despite the fact that evaluating the application’s impact on triage accuracy and response time is essential for assessing its clinical effectiveness, but these aspects are part of another study designed to quantitatively measure decision-making accuracy and efficiency in real-world scenarios, which regarding limitations we had in current study, can be conducted in future.

At the last phase of the application development process, the latest version of the application was evaluated for its usability. In order to initiate the evaluation process, a standard questionnaire known as the Mobile Health Application Usability Questionnaire (MAUQ) was used. Specifically, the version designed to evaluate mobile applications used by healthcare providers, developed by Zhou et al.³⁰, was employed. The final edition of the questionnaire comprises 7 questions asking user’s demographic background and a total of 18 statements which aimed to evaluate 3 criteria regarding app’s usability. First 5 statements were investigating the “Ease of Use”, 7 statements were about app’s “Interface and Satisfaction”, and the last 6 ones were included to explore the “Usefulness” of the application. The questionnaire was in a 5-point Likert scale, which 5 signifies a complete agreement and 1 means the user completely disagrees with the statement. Two important factors concerning a functional questionnaire are its validity and reliability. In this study, the validity was confirmed by medical informatics experts’ opinion and the reliability was calculated and confirmed in previous studies³⁰. The reliability of the questionnaire was re-examined and the internal correlation coefficient of the questions was calculated (Cronbach’s alpha = 0.836). Thirty-two users were voluntary participated in the evaluation proceedings. The inclusion criteria were as follows: having more than three years of working experience in pre-hospital emergency settings, being an active paramedic or EMT technician, being employed by Tehran Emergency Medical Services. A facilitator, who was in touch with on-duty paramedics in different government- owned emergency bases, facilitated the recruitment of participants. We excluded participants from the private sector, as the main service of the private sector in this context in Iran is the transfer of the injured rather than applying the triage approach. In addition, we did not include the female participants, as they did not meet the working experience criterion. Since the EMS system in Iran already employs mobile application for its assignments, all the users were well familiar with mobile technology. First, the participants were provided with necessary explanation regarding the study’s goal as well as adequate instruction on how the application works and what are the main features. Then, they had a chance to work with it for a short time and inspect all the features. At the end, they were asked to complete the questionnaire. Not to mention that they were all assured of the confidentiality of the collected data. After that, the data collected from the papers were analyzed using SPSS 27.

To facilitate the reporting of the results, a usability evaluation scale was considered, in which the range of mean values was as follows: 0–3 (not acceptable), 3.1-4 (relatively acceptable) and 4.1-5 (acceptable).

Findings

Findings related to application development

After extracting the functional requirements from literatures, these were confirmed by experts and used as a basis for designing and developing the application (Table 1). The overall use-case diagram of the application is presented in Fig. 1.

Table 1.

Functional requirements of the application

No.	Functional requirement	Description
1	Sign in	The application should enable user to create an account by entering personal, organizational and security information
2	Log in	The application should enable user to enter the application using their username and password
3	New Case	The application should provide user with in-depth categorized form to collect data
4	Triage Level	The application should provide user with the most appropriate triage level according to data collected from the patient
5	Navigation	The application should enable user to search the destination hospital and navigate the best possible route
6	File Transfer	The application should allow user to transfer case report files electronically
7	History	The application should enable user to have access to previous case reports
8	Education & Information Resources	The application should enable user to have access to educational and information contents like protocols and instructions
9	Case Report	The application should provide user with a well-structured and integrated case report concluded from the information forms

Fig. 1.

Use-case diagram of the application.

Provided features and functions, from buttons, interface design to informational content were all based on requirements extracted from literature, questionnaires completed by paramedics and experts’ opinions. Since this application is designed to be used by paramedics in the field, working under critical circumstances, features, content selection and design were aligned with their professional needs and job nature to provide them with utmost access in short time.

Discussing functional requirements, apart from those which are mainly related to triage and reporting function, educational and informational functions are considered in this application. Since both newly graduated paramedics and experienced ones are the application’s users, and the fact that learning is a dynamic lifelong process, users need to review and update their knowledge on a regular basis and have access to the latest guidelines and relevant educational contents. Thus, according to the requirements extracted from the questionnaires in development phase and literatures, educational and informational resources were included in the application.

Given that the application’s end users are Persian-speaking paramedics employed by Tehran EMS in Iran, the application’s default language is set to Persian.

The application’s main services are as follows:

• Providing users with an exhaustive detailed forms to collect data related to patient and the emergency situation. Due to fragile patient condition and paramedics’ work nature, they have to provide patients with immediate emergency care as fast as possible. So, there will be a narrow window to generate reports while in action. Hence, by these forms, paramedics are mainly required to choose items instead of typing the whole description.

• Providing users with human body and car graphs to illustrate injury regions and crash mechanism.

• Providing users with appropriate triage level based on RTS.

• Navigating routes with the help of GPS technology and Google Map application in order to find the optimum path from the incident cite to the destination hospital.

• Converting all information forms into a comprehensive case report in PDF format. Through this service, data entered in the new case forms are replaced in a structured, integrated report template with all graphical and numerical data included. So, individual users will not need to type and construct their own report differently.

• Online file transferring via platforms installed in the mobile device. Worth to mention that by the time of conducting current study, the application was not integrated with the HIS of hospitals in Tehran. But in case it gets ratified officially by Tehran EMS, reports can be transferred to HIS on a real-time basis to share data related to patient and their medical needs with providers at the hospital in advance to arrival.

• Representing users all latest case reports and allow them to organize items using an ascending/descending date filter.

• Providing users with educational and information contents in triage and prehospital care domains, including protocols, standards, triage systems and flowcharts.

• SOS direct call button.

Regarding privacy and policy issues, and to ensure the patient and provider’s confidentiality, a login/sign-in function was designed. Also, since paramedics are not allowed to use their own personal mobile phones for conducting case reports, and they have to work just with devices registered by Tehran EMS, at the beginning of each shift they need to sign in to the app with their specific username/password to let the supervisors identify who is on the shift and responsible for the outcomes, and which devices are being used throughout the shift assignment.

Hands-free functionality is a privilege that can assist paramedics when their hands are occupied with providing treatment to patients at the scene and on route. However, due to legal restrictions, no photos, videos, or even voice recordings are permitted from the scene or of the patient. Therefore, unless approved by legal authorities, this function has been excluded from the design process.

The class diagram of the application was provided in Supplementary Fig. 1.

Discussing application’s different sections and services, there is a horizontal bar at the bottom of the screen gives access to different parts of the application. On the home page, the entry button for new case forms is displayed as well as the latest three case reports, and if the user is willing to organize reports, an ascending/descending filter is embedded (Fig. 2).

Fig. 2.

Application’s home page.

In case, user needs to review or study subjects related to triage such as triage systems, triage principles, triage regions, prehospital care and many other technical terms, there is a button named education on the horizontal bar which contains all the aforementioned contents (Fig. 3).

Fig. 3.

Education resources.

As education resources, there is a section dedicated to guidelines, protocols and instructions any paramedic needs to be aware of. In addition, the navigation service can be found in this section which is an extension of the hospital lists and their contact information (Fig. 4).

Fig. 4.

Information resources.

The profile icon displays user’s personal information as well as contact information and application’s night mood switch.

Findings related to application evaluation

All 32 participants involved in the evaluation were men, ranging from the minimum age of 25 to 55 years old as the oldest participant, with the average age of 41.78 ± 8.335 years. The least experienced one had been working in EMS system for three years and the most experienced paramedic had been on the job for 26 years (Table 2). Almost all of them had relevant educational background. More than 59.4% had bachelor, 9.4% graduated with a master, and 25% had associate degree. Regarding the field of study, more than half of the participants had been educated in Medical Emergencies (56.3%), 28.2% had Anesthesiology and Nursing degree (Table 2).

Table 2.

Demographic details of participants involved in the evaluation process

	Variable	Number	Percentage
Age	< 30	3	9.4
	30–40	13	40.6
	41–50	11	24.4
	> 50	5	15.6
Work experience in EMS (year)	< 6	2	6.3
	6–10	5	15.6
	11–20	15	46.9
	> 20	10	31.3
Academic degree	Associate	8	25
	Bachelor	19	59.4
	Master	3	9.4
	Other	2	6.3
Field of study	Anesthesiology	3	9.4
	EMS	18	56.3
	Health in emergencies & disasters	1	3.1
	Nursing	6	18.8
	Other	4	12.5
Rank	EMT Paramedic	20	62.5
	EMT Intermediate	7	21.9
	EMT Basic2	5	15.6

After analyzing the extracted data in SPSS, the total mean score for the usability of the application was 4.12 ± 0.59. Among the usability criteria, the ease-of-use achieved the highest mean score of 4.35 ± 0.47, followed by usefulness at 4.12 ± 0.65, and interface and satisfaction at 3.89 ± 0.79 (Table 3).

Table 3.

Descriptive statistics of the evaluation criteria

	Mean ± SD	Min	Max
Ease-of-use	4.35 ± 0.47	3.4	5
Interface & satisfaction	3.89 ± 0.79	2	5
Usefulness	4.12 ± 0.65	2.5	5
Total	4.12 ± 0.59	-	-

Discussion

The importance of prehospital cares including triage, and its pivotal impact on patients’ health condition, as well as medical diagnosis accuracy and precision, followed by further treatment plans, makes it necessary for EMS systems to employ technologies such as mobile applications to collect and process data regarding medical emergencies and so-called incidents. Therefore, a mobile application was developed and evaluated to facilitate paramedics and EMT personnel throughout their dispatch calls, collecting data and form a comprehensive report so that patients can receive the utmost treatment and interventions they need. According to the findings, the overall level of usability of the application was 4.12 ± 0.59 which was assigned as acceptable, and among all usability criteria, the satisfaction and interface had the lowest mean score. Among factors affecting interface & satisfaction criteria, time and real-world settings are debatable. Since the participants were all active-duty paramedics at the time of the evaluation, and were obliged to use already registered application called Asayar, they didn’t have the opportunity to employ the proposed application in real dispatch calls, so subsequent assessments weren’t possible. Moreover, they weren’t fully familiar with app’s components and structure, so it seemed to be time consuming using it in order to accomplish tasks at first glance. In their study, Lee et al. indicated that services such as data collecting, including vital signs and clinical data could improve the quality of care, a potential for data driven electronic triage system and increase the proportion of patients who receive care²⁰. In the proposed application, all data and information, from patient’s health condition including their clinical impressions, vital observations and history to incident information, were all documented in 14 classified forms accompanied with visual displays of human body and car to illustrate trauma and injury regions, and crash points respectively. This service was similarly provided in an application by Zoll cooperation, named RescueNet e-PCR, which assisted paramedics collecting patient’s data in a comprehensive platform²⁸.

In a study by Sutham et al., along with services provided for rapid triage, educational content was introduced to help both patients and emergency staff stay updated and gain sufficient skills in the medical emergency filed³¹. In this regard, educational and informational contents were of main services in the present study, which was implemented based on literature, guidelines, protocols and users’ educational needs.

In an application developed by General Devices named E-bridge, case summary was a main function, which generated a concise report from data collected³². Similarly, in the current study, case report was provided as a main service, which contains all data entered by user in an integrated and comprehensive structure, representing the appropriate triage level too.

There are numerous health applications focusing on a specific field of medicine. Like FAST-ED developed by Nogueira et al., which was mainly designed to provide cardiac patients with prehospital care and assessment²². In the current study, informational contents in new case forms were designed in a way that all possible scenarios including cardiac emergencies can be supported in detail.

In medical emergency transports, time plays a significant role in mission success and patient outcome. Hence, it is crucial for paramedics to find the optimum route from the incident site to the hospital. In studies by Wu et al. and Nimmolrat et al. GPS technology was exerted to navigate the best way through conventional road traffic^8,33. Similarly, current study proposed a navigation service using GPS technology and Google Map software within the app environment to navigate the best path available with regard to hospital specialty and patient’s needs.

In proposed application, instead of using paper consent forms and mission outcome documentation, paramedics had access to digital forms as “Continuity of Care & Refusal” to report the end of the call and consequent results. Moreover, in situations where paramedics need to make a quick call over dispatch center, a SOS button was embedded in the app’s environment that directly dials medical emergency phone number.

According to evaluation results of the current study, users were satisfied with the application with an overall mean score of 4.12 ± 0.59. Among all evaluation criteria, ease of use had the highest mean score (4.35 ± 0.47) which indicated that users were satisfied with consistency of navigation, moving between pages and components, and learnability. Likewise, in a study by Mortezaei et al.³⁴, who used MAUQ to evaluate the usability of a mobile application developed for Albinism self-management, results showed a high mean score (6.01 ± 0.89 in 7-point Likert scale) for ease-of-use criterion.

A positive point of the current study could be its focus on developing a mobile application for prehospital triage, employed by EMS operational personnel, optimizing their performance and improve patient care. In addition, the functional requirements of the application that initially extracted from the literature and the questionnaire were discussed with experts, so, these two steps resulted in design and development of key services provided in the application. However, there were some limitations to the current study. First, the application was developed for devices running the Android operating system, which affected the use of the developed application for devices with other operating systems such as iOS. In addition, a proper access to EMS personnel was restricted by the nature of their career and organizational policies. Moreover, the time needed for evaluation process was limited due to the fact that participants were all on call, and they would have been dispatched any moment. Regarding regional variability, informational content and organizational protocols, the application was developed only for the Tehran EMS system and the wide adoption of such as system in other regions requires further developments. Additionally, since Tehran EMS already employs a mobile application, it was not possible to fully test the application in real world settings. In the current study we focused on usability testing of the application and obtaining a deeper understanding of application and its impact require the application of other evaluation methods such as cognitive walkthrough (CW) and think aloud (TA). Due to contextual and time limitations, we were unable to apply such approaches. Therefore, further evaluation of the application in terms of usability and impacts over a longer time span and broader population is required.

Conclusion

The mobile application developed and evaluated in this study has the potential to be employed in prehospital settings in order to help paramedics and EMT personnel as a tool to collect and report data related to medical emergencies and provide patients with optimum care and assign each case an appropriate triage level. To order the evaluation criteria based on the mean scores extracted from questionnaires, the ease of use and usefulness obtained the highest value and were considered as acceptable, whereas the interface and satisfaction had the least mean score and was regarded as relatively acceptable. These findings indicate the importance of time and real operational setting in evaluation process and application’s demonstration. In addition, over time and with the utilization of the application in real operational settings, further evaluation is suggested to improve the application’s performance, in particular the interface components.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Author contributions

E.A. participated in the conceptualization and design, application modeling and development, formal analysis, and writing the initial draft. H.E. and R.R. participated in the conceptualization and design, formal analysis, drafting and editing. M.S. supported content development of the application and facilitated users’ recruitment for testing the application. All authors have read and approved the final manuscript.

Data availability

The data underlying this article cannot be shared publicly due to the privacy of individuals who participated in the study and the institutional policies. The questionnaires used for the requirement analysis and usability testing are available upon request from the corresponding authors.

Declarations

Competing interests

The authors declare no competing interests.

Ethical approval

In this study, all methods were carried out in accordance with relevant guidelines and regulations. The study was approved by the Research Ethics Committee of Shahid Beheshti University of Medical Sciences. (Approval ID: IR.SBMU.RETECH.REC.1402.870)

Consent to participate

All participants were fully informed of the aims and scope of the study, and informed consent was obtained from all subjects. To maintain confidentiality, no personal or identifiable data were reported in this study and pseudonyms were used when illustrating the screens of the application.