Predicting Firefighter Injury and Entrapment in Urban Firefighting Operations: An Investigation Into the Effectiveness of Modified Fire Time Stages and Machine Learning

ABSTRACT

Background and Aims

Urban fires pose a significant threat in terms of property damage and potential loss of life. Firefighters play an important role in managing these incidents, so their safety is a top priority for fire departments and emergency responders. The purpose of this study is to predict the probability of firefighter injury and entrapment before dispatching the operational team or the initial stages of the urban firefighting operations.

Methods

In this study, we compare the performance of eight machine learning algorithms in predicting the occurrence of firefighter injuries and entrapment during urban fire incidents. We use data from the Fire Department Operations and Management System (FOMS) of Mashhad's city fire and safety services organization. Specifically, we assess the effectiveness of the generated models through five stages, two of which use additional features calculated using the built‐in FOMS functions.

Results

We found that the Multi‐Layer Perceptron and Kernel Naive Bayes algorithms achieved the highest predictive accuracy for the occurrence of firefighter injuries and entrapments, 96.7% and 96.0%, respectively.

Conclusions

The generated models can help to reduce the incidence of injuries and entrapments among firefighters by providing the decision‐makers with early prediction of the risk factors associated with an ongoing urban firefighting operation.

Summary

• Urban firefighting operations pose significant risks of injury and entrapment to firefighters, and predictive modeling has the potential to mitigate these risks.

• This study demonstrates that machine learning algorithms, particularly Multi‐Layer Perceptron and Kernel Naive Bayes, can predict firefighter injuries and entrapments with high accuracy (96.7% and 96.0%, respectively) by incorporating modified analysis and additional operational features.

• Implementing these models as part of a Decision Support System can improve operational safety, reduce firefighter injuries, and enhance emergency response strategies.

1. Introduction

One of the most serious threats to human life is urban fire. Fires cause severe emotional and social effects, resulting in disabilities, injuries, and death [1, 2, 3]. According to the 2021 report by the International Association of Fire Services, 3.1 million fires have occurred in 2019 in 34 countries, which account for 15% of the world's population. These fires resulted in 16,800 deaths (0.5 death per 100 fires) and 47,900 thousand injuries (1.5 fire injuries per 100 fires) [4]. The report published by the World Health Organization (2016) estimated that 180,000 people died from burns caused by fires. The vast majority of these fatalities occurred in low‐and middle‐income countries [5]. Fires cause severe damage to buildings, urban infrastructure [6, 7], and the biophysical environment by releasing carbon dioxide that contributes significantly to climate change, negatively impacting people's health [8, 9].

Interventions to reduce the negative consequences of fires are categorized into preventive and postaccident measures. Preventive interventions aim to reduce the likelihood of fires through safe building design [10], smoke detectors [11], fire safety education [12], and prediction models [13, 14, 15, 16, 17]. Postaccident interventions focus on mitigating fire damage by shortening response times [18], using decision support systems (DSS) for fire management [19], and simulating fire scenarios to improve services [20]. Notably, using machine learning techniques to predict the fires' negative consequences is one of the most important preventive interventions [13, 14, 15, 16, 17].

From 1993 to 1998, fire mortality data from the New Zealand Fire Service Fire Information Recording System were collected and used to forecast the number of fatal residential fires in geographical mesh blocks. This study found an association between deprivation and the rate of unintended fatal residential fire injuries [13]. Similarly, a study by Fang et al. proposed a data‐driven approach to identify fire development stages in residential room fires using machine learning methods. This approach involved analyzing various fire parameters, such as heat release rates and smoke production, to classify the fire's progression more accurately [14].

Further, an unsupervised self‐organizing feature map neural network was employed to cluster and assess risk levels of fire incidents. This model clustered fire incidents into five risk levels: very low, low, moderate, high, and very high. It was evaluated using data collected between 2000 and 2007 from Toronto's fire incidents [15].

Firefighters face numerous risks, including burns, smoke inhalation, extreme temperatures, and toxic gases. According to the US National Fire Protection Association, there were 60,750 firefighter injuries in 2021 alone [16]. One study utilized generative adversarial neural networks (GANs) to predict flashovers by analyzing video streams from firefighters' body cameras. These neural networks enhanced dark fire and smoke patterns, allowing for the prediction of flashovers up to 55 s before they occurred [17]. Another study employed support vector machines to estimate fire arrival times using satellite data, achieving high accuracy in predicting when and where a fire would spread, which is crucial for planning and resource allocation during firefighting operations [21].

Another study used statistical analysis and machine learning to analyze data from the Canadian National Fire Information Database from 2005 to 2015 [22]. The following features were identified as the main factors that impact firefighter injuries: Firefighter‐Helmet Worn at Time of Injury, Firefighter‐Helmet Line Used at Time of Injury, Firefighter‐Coat (Turnout) Worn at Time of Injury, Firefighter ‐ Boots Worn at Time of Injury, Fire Fighting Years of Experience, Age of Victim, Height Firefighter, and Weight Firefighter. Also, the following features were identified as influential determinants: Initial detection, Building height, Ground floor area, Major Occupancy Group, Energy causing ignition (a form of heat), Fuel or energy associated with igniting object, Act or Omission Group, Material First Ignited Group, Sprinkler protection, Manual fire protection facilities, Area of Origin Group, Igniting Object Group, Level of Origin, Automatic fire detection system, Number of occupants, Property Classification Group, Action taken, and Method of Fire Control & Extinguish Group [22].

In addressing the critical challenge of mitigating risks associated with firefighter fatalities and injuries, it is evident that there exists a notable gap in the current understanding and application of predictive models for proactive use by fire departments and emergency responders. While various strategies are employed, the need for more refined and advanced methodologies to anticipate and manage firefighting incidents is apparent. This study aims to fill this gap by developing and implementing machine learning models to enhance the prediction and management of firefighting incidents, thereby reducing injuries and fatalities among firefighters.

The purpose of this study is to address these issues and provide the following primary contributions: (1) Advanced Forecasting Models: We have developed forecasting models that successfully predict two critical time points related to firefighting incidents earlier than their actual occurrence and introduced two modified stages of fire incidents. This predictive capability allows us to use these early indicators as input variables for a comprehensive risk assessment model, improving the timeliness and effectiveness of firefighting responses. (2) To improve the accuracy of our predictive models, we integrated additional features into the modified stages of fire incidents. For example, we utilized weather conditions and location data to more accurately calculate the necessary response times for firefighting teams. These enhancements enable more informed and timely deployments, reducing risks and improving safety outcomes in urban firefighting operations. (3) We employed eight diverse machine learning models across five fire incident stages, focusing particularly on the two stages enhanced through early prediction. These advanced models, equipped with additional features, demonstrated high and consistent accuracy in forecasting the risk of injury and entrapment. This comprehensive approach ensures robust and reliable predictions. (4) The study aims to enhance firefighter safety by predicting injury and entrapment risks in advance. By providing these forecasts, firefighting teams can be better equipped and prepared, potentially preventing hazardous incidents and improving overall safety. The workflow of the study is shown in the Figure 1.

Figure 1.

Workflow of the study.

Our goal is to enhance the safety of firefighters during operations by predicting the probability of injury and entrapment of firefighter. We achieve this by analyzing data on past operations, predicting the features that exist in the model but are not available at the beginning of the operation, and introducing new features to the existing data set. By furnishing the operation team with this predictive information before dispatch or at the initial stages of an operation, we aim to minimize adverse incidents, thereby promoting firefighter safety and mitigating risks in firefighting operations.

2. Materials and Methods

2.1. Data

This section discusses the data utilized in this study as well as how it was preprocessed. We rely on data stored in Mashhad city's fire and safety services organization's Fire Department Operations and Management System (FOMS). FOMS is a complete system for managing fire department operations that includes a suite of tools for incident management, personnel management, training and development, and safety and risk management. It also has a web‐based interface for simple access and data sharing across fire departments.

We analyzed FOMS data gathered between March 2017 and September 2021. During this time, 89,461 firefighting operations data points were collected, with each operation described by 45 features. Supporting Information S1: Table A1 describes the 45 features.

Each firefighting operation has five stages: T103, T104, T105, T106, and T107. These stages are detailed below.

2.2. Time to Announce to the Operational Team (T103)

When a person calls the fire and safety services organization, the call center department assesses the information provided to determine the need to dispatch an operational team. If an operational team is needed, the call center department collects the address of the incident site and uses FOMS, which records the status of the stations and operational teams, to locate the nearest available team. The operational crew is then dispatched from there. Supporting Information S1: Table A1 explains the information recorded at this stage.

2.3. The Arrival Time of the Operational Team (T104)

When the operational team arrives at the scene of the incident, they use the software on their mobile device to determine the geographical coordinates (longitude and latitude) and time of arrival. This data is subsequently communicated to the FOMS through the internet. Supporting Information S1: Table A1 explains the data collected at this stage.

2.4. The Time of Completion of the Operation (T105)

Following the operation, the commander uses software to determine the exact time of completion, which is subsequently relayed to the organization's central system over the Internet. The data recorded at this stage includes the completion time of the operation (T105). Supporting Information S1: Table A1 explains the information collected at this stage.

2.5. The Time of Return to the Station (T106)

Following the completion of the mission, the operational team will normally conduct an equipment inspection before returning to the fire station. Furthermore, during this time, persons who have been engaged in the accident are usually assisted, as well as any inquiries that may emerge. The operation commander uses software to report when the operational team is ready to return to the fire station and broadcasts it via the central internet. The time of return of the operational crew to the fire station (T106) is the data recorded at this stage. Supporting Information S1: Table A1 explains the information collected at this stage.

2.6. The Time of the Operational Team's Arrival at the Station (T107)

The operational team's arrival time at the station is recorded and relayed to the central system via a software program installed on the operation commander's device. This data is critical for managing firefighting operations and assuring their effectiveness. The recorded data (T107) is used as a reference for future operations management. Supporting Information S1: Table A1 displays the data collected during this stage.

Additional data, depending on the type of operation, is also recorded. In the instance of a fire, the commander records the material that started the fire as well as the substance that contributed the most to the volume of fire. The operation commander provides a short description of the operation in two to three paragraphs. Each operation also records firefighter deaths, injuries, and entrapment. Recording information during the operation (stages T104 to T106) can disrupt the operation. In these stages, only the timing of each stage is documented. Consequently, most of the operation‐related information is entered once the team returns to the station.

2.7. Firefighter Injury and Entrapment As the Label Variables

Supporting Information S1: Table A1's last three elements are dependent variables. Because firefighter deaths occurred only in two firefighting operations, we omitted the death variable from our analysis. As a result, we have two dependent variables (labels). For each, we will create a machine learning model.

2.8. Data Preprocessing

Data preparation is the critical process that determines the quality of the resulting machine learning models. Cleaning and validating data, dealing with missing values, data reduction, feature selection, feature normalization, and discretization are all part of the process [23]. Preprocessing data helps to guarantee that the data is ready for subsequent analysis [24, 25, 26].

During the data collection and input phases, the data collected from FOMS (version 2) were checked for correctness and completeness. We looked for missing data and found just 240 records containing them. We rejected all of them because they accounted for barely 0.3% of all data. This left us with 89,221 valid records to work with when developing prediction models.

Following that, we used feature selection to see if we could reduce the dimensionality of the data from the original 42 + 2 class labels. We employed forward feature selection (FFS) method on the 42 features; It starts with an empty set of features and adds features until the desired performance of a classification model is attained. The feature that improves performance the most is introduced first, and the procedure is repeated until no more improvement is obtained. Feature selection effectively addresses problems in high‐dimensional datasets by removing irrelevant and redundant data, thereby reducing computation time, improving learning accuracy, and facilitating a better understanding of the learning model and data [27, 28].

We also interviewed firefighter experts after the feature selection process was completed, to ensure their agreement with the algorithmic decisions for features elimination. While they agreed in most cases with the algorithm, they recommended that we keep some of them because they are crucially important in firefighting operations. Table 1 displays the outcomes of the feature selection procedure (fourth column) as well as expert opinions (fifth column). As can be seen, even though the FFS requested that a feature be removed (marked as NO), it was retained in the model (shown as “YES”). As a result, only 23 features were kept for model construction.

Table 1.

Explaining 23 features used in model building (marked bold). The last two features are class labels.

No.	Variable name	Stage collected	Features used (YES) in model building	Features indicated by FSS for elimination (NO)
1	Operational Code	T103	NO	NO
2	Malicious reporting	T103	NO	NO
3	Announcement time	T103	YES	YES
4	Shift work	T103	YES	YES
5	Fire station	T103	YES	YES
6	Incident location	T103	NO	NO
7	Weather condition	T103	YES	YES
8	Temperature	T103	YES	YES
9	Wind direction	T103	YES	YES
10	Wind speed	T103	YES	YES
11	Wind degree	T103	YES	NO
12	Horizontal field of view	T103	YES	NO
13	Air pressure	T103	YES	NO
14	Air humidity	T103	YES	NO
15	Phone number	T103	NO	NO
16	Last name	T103	NO	NO
17	Redevelopment code	T103	NO	NO
18	Municipal district	T103	YES	YES
19	Arrival Time	T104	YES	YES
20	Longitude	T104	NO	NO
21	Latitude	T104	NO	NO
22	Distance	T104	YES	YES
23	Completion time	T105	YES	YES
24	Return time	T106	YES	YES
25	Operational team's arrival at the station	T107	YES	YES
26	Type of service (a three‐level hierarchical structure)	T107	YES	YES
27		T107	YES	YES
28		T107	YES	NO
29	Type of incident site	T107	YES	YES
30	Two‐level hierarchical structure	T107	YES	YES
31	First source of ignition	T107	YES	YES
32	Primary combustible material	T107	YES	YES
33	Heat source	T107	YES	YES
34	Intentional/unintentional	T107	NO	NO
35	Description	T107	NO	NO
36	Additional operation information	T107	NO	NO
37	Date and time of operation completion	T107	NO	NO
38	Emergency team	T107	NO	NO
39	Police	T107	NO	NO
40	Vehicles	T107	NO	NO
41	Members	T107	NO	NO
42	Commander	T107	NO	NO
43	Firefighter death			Class Label
44	Firefighter injury			Class Label
45	Firefighter entrapment			Class Label

3. Methods

This section is divided into two subsections. The first introduces the ML algorithms used to predict firefighter injury and entrapment in urban firefighting operations. The second describes methods used in evaluating the generated models.

The most challenging aspect of data is its unbalanced nature. In other words, the number of firefighting operations in which a firefighter has died, been injured, or trapped is small. There are two approaches to dealing with imbalanced data: data‐level and algorithmic‐level. In the first approach, data are enlarged to create a balanced data set while in the other the models are modified to address issues associated with imbalanced data [29, 30, 31]. Data‐level techniques are widely used due to their efficiency and independence from predictive models [31, 32]. Data‐level strategies are divided into two categories, under sampling and oversampling. The former involves eliminating examples with large number of observations, while the latter involves replicating examples with a smaller number of observations. We used oversampling since it can balance class distributions without eliminating potentially critical majority examples [32].

K‐Nearest Neighbors [33], random forest (RF) [34], gradient boosted trees (GBT) [35], Naive Bayes [36, 37], multi‐layer perceptron (MLP) [38], H2O based deep learning [39], and AutoMLP [40] are used in this study to generate prediction models. In the Supplemental Machine Learning Models section, we briefly describe the supervised machine learning techniques used to generate the prediction models.

To evaluate goodness of the generated models we used cross‐validation and report the results in terms of accuracy. k‐fold cross validation is a method of evaluating a model by splitting the data into k subsets, and training the model k times, each time using a different subset as the testing set and the remaining k‐1 subsets as the training set [41]. The average of the k tests is then reported. k = 10 was used in this study.

Accuracy is defined as:

$$\mathit{Accuracy}=\frac{\mathit{TP}+\mathit{TN}}{(\mathit{TP}+\mathit{TN}+\mathit{FP}+\mathit{FN})}$$

The values of true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN) are obtained from a confusion matrix [42] shown in Supporting Information S1: Table A2.

TP are instances where the model correctly predicts a positive outcome. FP are instances where the model incorrectly predicts a positive outcome. TN are instances where the model correctly predicts a negative outcome. FN are instances where the model incorrectly predicts a negative outcome (see Supporting Information S1: Table A2).

3.1. Parameter Tuning

Parameter tuning in data mining is an essential step in optimizing the performance of machine learning models [43]. It involves adjusting the values of various parameters used in the machine learning algorithms to obtain a model that best fits the data. There are different techniques for parameter tuning. In this study we used grid search which involves testing different combinations of parameter values within a predefined range [44].

4. Results

4.1. Accuracy of Predictive Models

This section employs eight distinct machine learning techniques to predict two objectives across five stages of firefighting operations. Specifically, the models used in this study include MLP, AutoMLP, H2O based Deep Learning, Kernel Naive Bayes, GBT, k‐NN, RF, and Naive Bayes.

These models were applied at three different stages: T103 (Time to announce to the operational team), T104 (The arrival time of the operational team), and T107 (The time of the operational team's arrival at the station). In the model built at T103, only the data collected at this stage are involved. In the model built in T104, the data created in T103 and T104 were used in creating the model. And finally, T107, the data collected in stages T103 to T107 have been used in building the model. Because data are collected incrementally for each firefighting operations, the accuracy of the models that are built on the last stages is expected to increase. In other words, the model created with the data collected up to stage 107 should be the most accurate, while the one created with the data collected at T103 should be the least accurate.

4.2. Accuracy of Models Built Using T103 Data

The data collected in T103 was used to build the predictive models, which include 89,221 records described by 12 features (features 3, 4, 5, 7, 8, 9, 10, 11, 12, 13, 14, 18 in Table 1). The input data size for model building at this stage is 89,221 × 12. The outcome of this analysis is shown in Table 2.

Table 2.

Comparison of different models at stage T103 (average of 10‐fold CV).

Model	$\mathbf{Accuracy\; Avg}\pm \mathit{SD}$ Firefighters' Injury	$\mathbf{Accuracy\; Avg}\pm \mathit{SD}$ Firefighters' entrapment
Kernel Naive Bayes	$82.7\%\pm 1.29$	$\mathbf{91.6}\%\pm \mathbf{1.41}$
H2O based deep learning	$69.6\%\pm 6.68$	$\mathbf{89.4}\%\pm \mathbf{1.69}$
Random forest	$68.0\%\pm 1.31$	$\mathbf{86}\%\pm \mathbf{1.85}$
Naive Bayes	$72.9\%\pm 0.94$	$\mathbf{84}\%\pm \mathbf{1.41}$
AutoMLP	$\mathbf{85.4}\%\pm \mathbf{1.52}$	$\mathbf{89.4}\%\pm \mathbf{2.02}$
GBT	$81.3\%\pm 0.91$	$\mathbf{91.3}\%\pm \mathbf{1.42}$
MLP	$81\%\pm 1.39$	$\mathbf{89.5}\%\pm \mathbf{1.13}$
k‐NN	$77\%\pm 0.69$	$\mathbf{83.6}\%\pm \mathbf{2.79}$

According to Table 2, AutoMLP was the most accurate algorithm for forecasting the occurrence of firefighters' injuries. Still, Kernel Naive Bayes had the highest accuracy in determining whether or not a fireman was trapped.

To illustrate, Supporting Information S1: Figure A1 shows that the algorithms better predict firefighter entrapment than firefighter injury. Also notice that Kernel Naive Bayes algorithm performed best, while for predicting injury random AutoMLP was the best.

4.3. Accuracy of Predictive Models in T104

The data collected in T104 and T103 was used to apply the models, which include 89,221 records described by 12 features from stage T103 and 2 features from stage T104; 2 features are 19 and 22. In other words, the input data size for the prediction model at this stage is 89,221 × 14. The findings from these models are shown in Table 3.

Table 3.

Comparison of different models at stage T104 (average of 10‐fold CV).

Model	$\mathbf{Accuracy\; Avg}\pm \mathit{SD}$ Firefighters' Injury	$\mathbf{Accuracy\; Avg}\pm \mathit{SD}$ Firefighters' entrapment
Kernel Naive Bayes	$88.0\%\pm 0.65$	$\mathbf{93.1}\%\pm \mathbf{1.97}$
H2O based deep learning	$\mathbf{82}.\mathbf{3}\mathit{\%}\mathit{\pm }\mathbf{9}.\mathbf{89}$	$\mathbf{92.2}\%\pm \mathbf{3.59}$
Random forest	$67.6\%\pm 0.79$	$\mathbf{85.8}\%\pm \mathbf{2.11}$
Naive Bayes	$51.5\%\pm 6.59$	$\mathbf{84.9}\%\pm \mathbf{1.36}$
AutoMLP	$96.9\%\pm 0.55$	$\mathbf{89.4}\%\pm \mathbf{2.02}$
GBT	$89.6\%\pm 1.04$	$\mathbf{92.4}\%\pm \mathbf{1.71}$
MLP	$96.6\%\pm 0.85$	$\mathbf{92.6}\%\pm \mathbf{1.37}$
k‐NN	$70.0\%\pm 1.32$	$\mathbf{76.0}\%\pm \mathbf{3.01}$

AutoMLP was the most efficient algorithm for predicting firefighters' injury incidents. On the other hand, Kernel Naive Bayes showed the highest accuracy for entrapment.

To illustrate, Supporting Information S1: Figure A2 shows that indicates that the accuracy levels are more consistent in predicting firefighter entrapment. Additionally, the MLP Algorithm was identified as the most effective approach considering the accuracies for both objectives.

4.4. Accuracy of Predictive Models in T107

We have built the models using all 20 features, which includes data from stages T103, T104, T105, T106, and T107 In other words, the input data size to the prediction model is 89,221 × 25. The results are shown in Table 4.

Table 4.

Comparison of different models at stage T107 (average of 10‐fold CV).

Model	$\mathbf{Accuracy\; Avg}\pm \mathit{SD}$ Firefighters' Injury	$\mathbf{Accuracy\; Avg}\pm \mathit{SD}$ Firefighters' entrapment
Kernel Naive Bayes	$90.5\%\pm 0.78$	$\mathbf{96.0}\%\pm \mathbf{1.57}$
H2O based deep learning	$91.5\%\pm 4.31$	$\mathbf{94.9}\%\pm \mathbf{1.26}$
Random forest	$84.4\%\pm 0.7$	$\mathbf{92.9}\%\pm \mathbf{0.96}$
Naive Bayes	$69.8\%\pm 1.1$	$\mathbf{83.7}\%\pm \mathbf{2.36}$
AutoMLP	$97.5\%\pm 0.56$	$\mathbf{98.3}\%\pm \mathbf{0.87}$
GBT	$89.1\%\pm 0.73$	$\mathbf{94.2}\%\pm \mathbf{1.84}$
MLP	$\mathbf{97}.\mathbf{9}\mathit{\%}\mathit{\pm }\mathbf{0}.\mathbf{28}$	$\mathbf{97.1}\%\pm \mathbf{0.84}$
k‐NN	$88.9\%\pm 0.38$	$\mathbf{87.5}\%\pm \mathbf{1.51}$

Table 4 shows that all algorithms exhibit good accuracy, with some, such as MLP and AutoMLP, achieving highest accuracy levels for both entrapment and injury. This was expected as 25 features describing the data provide much more additional information. Achieving good accuracy in stage T107 is important as it establishes an optimum value. Overall, the results shown in Table 4 represent maximum attainable accuracies.

To illustrate, Supporting Information S1: Figure A3 shows that all algorithms exhibit good accuracy, with some, such as MLP and AutoMLP, achieving highest accuracy levels for both entrapment and injury. This was expected as 20 features describing the data provide much more additional information.

4.5. Modified Analysis

The accuracies of the models based on data from stage T103 (Time to announce to the operational team) were not high when compared to the models that used as input data from stages T104 (The arrival time of the operational team) and T107 (The time of the operational team's arrival at the station). However, the model developed based on stage T107 data (after the operation is completed), is not very useful for fire stations since it is essential to predict entrapment and injury at early stages of the firefighters operation. Another point is that the difference between models accuracies based on stage T104 and stage T107 data is small, which is understandable as a large amount of information has been collected up to and including stage T104.

Increasing accuracy of models developed based on early stages, namely, T103 and T104 data would be very helpful in mitigating negative consequences of entrapment and reducing the number of injuries. Therefore, we attempted to improve the accuracy of the model built using stage T103 data by: (a) adding and using additional features to the original fire department data (Supporting Information S1: Table A1), and (b) predicting data to be collected only in later stages (T104‐T107). This expanded, by new features stage T103 data is called T103Modified (T103M) data. The features added to stage T103 are also used to improve model at stage T104 is called T104Modified (T104M).

The following 5 features (a) were calculated by built‐in functions of the FOMS added to stage T103 (and T104):

• Hour of T103 (H_T103)

• Day of the week of T103 (DoW_T103)

• Daey of the month of T103 (DoM_T103)

• Season of T103 (S_T103)

• Day of the year of T103 (DoY_T103)

In addition, the following features were estimated in stage T103 and added to data of stage T103M (and T104M):

• Estimated geographical coordinates: obtaining accurate latitude and longitude data is difficult and not conducted until the operation commander reaches the site of the incident. To address this issue, we used GIS (in T103) to find a geographical point that corresponds to an incident address, allowing for a high‐accuracy estimate of the geographical location of the incident.

  This was added to the FOMS to provide a rough location of the incident to the call center staff when the address is typed in.

• Estimated time and distance traversed to reach the incident site: the data pertaining to this information is obtained at later stages of the incident response process. To supplement the information available at the operational station, we have incorporated five additional fields of data, which include: (1) the time taken from the fire station to the incident site, without taking into account the traffic conditions provided by the mapping systems, (2) the distance covered from the fire station to the incident site, without taking into account the traffic conditions provided by the mapping systems, (3) the time taken from the fire station to the incident site, considering the traffic conditions provided by the mapping systems, (4) the distance covered from the fire station to the incident site, considering the traffic conditions provided by the mapping systems, (5) the geographical distance between the starting point (fire station) and the destination (incident site), represented as a direct line. These additional fields of data were obtained by utilizing estimated geographical coordinates and using mapping systems. These features are used in stage T103M. Considering that these data are collected at T104, therefore, the estimated values at T104M will not be used and the actual values will be used.

• Estimated difference between times of T103 and T104: this has also been added to the data. This feature is used in stage T103M. Considering that this data are collected at T104, therefore, the estimated value at T104M will not be used but the actual value will be used.

• Type of incident and location (collected data): call center staff are trained to determine type of incident (the first level of three‐level structure) and location (the first level of two‐level structure) by asking questions of the caller (at stage T103). This information is crucial in determining the appropriate response and resources required to address the incident effectively. Having accurate and detailed information about the type of incident and its location also assists in prioritizing responses and ensuring safety of the responding personnel. These features are used in stages T103M and T104M.

The list of added fields to stages 103 and 104 is shown in Supporting Information S1: Table A3.

Features 1–5 are used in T103M and T104M. Features 6–11 are estimated in T103. These features are obtained at T104, so these estimated values are used only in T103. In T104 the actual values of features 8, 9 and 11 are used. Features 6, 7 and 10 have no meaning in 104, so they are not used in 104. Features 12 and 13 are determined by the operation commander in T107, but call center staff are trained to determine them.

4.6. Accuracy of Models in Stage T103m

Using these additional features, the models were built for stage T103M. The features used were the original 12 features collected in T103, plus 6 estimated features (features 6, 7, 8, 9, 10, 11), and 7 additional features (features 1, 2, 3, 4, 5, 12, 13). In other words, the input data size to the model building is now 89,221 × 25. The outcome of this analysis is shown in Table 5.

Table 5.

Comparison of different models at stage T103M (average of 10‐fold CV).

Model	$\mathbf{Accuracy\; Avg}\pm \mathit{SD}$ Firefighters' Injury	$\mathbf{Accuracy\; Avg}\pm \mathit{SD}$ Firefighters' entrapment
Kernel Naive Bayes	$89.63\%\pm 0.91$	$\mathbf{95.96}\%\pm \mathbf{1.61}$
H2O based deep learning	$84.00\%\pm 7.12$	$\mathbf{95.08}\%\pm \mathbf{1.81}$
Random forest	$66.02\%\pm 1.54$	$\mathbf{92.88}\%\pm \mathbf{1.63}$
Naive Bayes	$68.09\%\pm 0.97$	$\mathbf{86.96}\%\pm \mathbf{2.64}$
AutoMLP	$96.56\%\pm 0.36$	$\mathbf{95}\%\pm \mathbf{1.51}$
GBT	$89.9\%\pm 0.55$	$\mathbf{92.31}\%\pm \mathbf{1.82}$
MLP	$\mathbf{96}.\mathbf{67}\mathit{\%}\mathit{\pm }\mathbf{0}.\mathbf{69}$	$\mathbf{95.5}\%\pm \mathbf{1.46}$
k‐NN	$72.29\%\pm 0.91$	$\mathbf{78.00}\%\pm \mathbf{1.97}$

As indicated in the Table 5, MLP was found to be the most effective algorithm for predicting the occurrence of firefighters' injuries, whereas Kernel Naive Bayes had the highest accuracy in determining whether a firefighter was trapped or not. To facilitate a better understanding of the results, Supporting Information S1: Figure A4 is provided in appendix and Figure 2 provided below.

Figure 2.

Differences in accuracy T103 and T103M. (A) Differences in accuracy T103 and T103M for firefighter injury. (B) Differences in accuracy T103 and T103M for firefighter entrapment.

The Figure 2A and Figure 2B show differences in the algorithms' performance between the stage T103 data and the T103M data.

As illustrated in the Figure 2B, the majority of the algorithms show a noticeable improvement in their accuracy values. The algorithms that have improved the most are H2O based Deep Learning, MLP, and AutoMLP, with an increase of 20.64%, 19.36%, and 13.09% in accuracy, respectively. Additionally, Figure 2A clearly indicates that, except for k‐NN, all other algorithms have shown an increase in their accuracy values. The algorithm with the highest increase in accuracy is RF, with an increase of 7.95%.

4.7. Accuracy of Models in Stage T104m

After implementing the modifications discussed earlier, the models were built using 14 features collected in stages T103 and T104, plus 10 additional features (features 1, 2, 3, 4, 5, 8, 9, 11, 12, 13). In other words, the input data size to the prediction model at this new stage is 89,221 × 24. The outcome of this analysis is shown in Table 6.

Table 6.

Comparison of different models at stage T104M (average of 10‐fold CV).

	$\mathbf{Accuracy\; Avg}\pm \mathit{SD}$ Firefighters' Injury	$\mathbf{Accuracy\; Avg}\pm \mathit{SD}$ Firefighters' entrapment
Kernel Naive Bayes	$89.56\%\pm 051$	$\mathbf{96.46}\%\pm \mathbf{1}\mathit{z}.\mathbf{48}$
H2O based deep learning	$82.60\%\pm 12.88$	$\mathbf{95.85}\%\pm \mathbf{1.2}$
Random forest	$66.24\%\pm 0.93$	$\mathbf{93.73}\%\pm \mathbf{2.28}$
Naive Bayes	$64.5\%\pm 1.18$	$\mathbf{88.00}\%\pm \mathbf{2.81}$
AutoMLP	$\mathbf{97}.\mathbf{40}\mathit{\%}\mathit{\pm }\mathbf{0}.\mathbf{43}$	$\mathbf{96.85}\%\pm \mathbf{1.07}$
GBT	$89.47\%\pm 1.19$	$\mathbf{95.12}\%\pm \mathbf{1.78}$
MLP	$96.64\%\pm 0.63$	$\mathbf{97.69}\%\pm \mathbf{1.2}$
k‐NN	$75.29\%\pm 0.73$	$\mathbf{81.04}\%\pm \mathbf{3.27}$

As indicated in the Table 6, Auto MLP was found to be the most effective algorithm for predicting the occurrence of firefighters' injuries, whereas MLP had the highest accuracy in determining whether a firefighter was trapped or not. To facilitate a better understanding of the results, Supporting Information S1: Figure A5 is provided in appendix and Figure 3 is provided below.

Figure 3.

Differences in accuracy T104 and T104M. (A) Differences in accuracy T104 and T104M for firefighter injury. (B) Differences in accuracy T104 and T104M for firefighter entrapment.

The Figure 3A and Figure 3B show differences in the algorithms' performance between the stage T104 data and the T104M data.

The results shown in Figure 3A and Figure 3B indicate that the accuracy of the models has generally increased after new features were added. However, for firefighter injury prediction, the accuracy of the RF algorithm and BGT has decreased. Notwithstanding, this algorithm has shown the greatest improvement for the other objective. The Naive Bayes algorithm has achieved the highest increase in accuracy for predicting the occurrence of firefighter injury, improving by approximately 25.17%. The RF algorithm has achieved the highest increase in accuracy for predicting the occurrence of firefighter entrapment, improving by approximately 9.23%.

4.8. Comparison of All Algorithms in All Stages

In this section, we provide comparison of all the algorithms. First, we present spider plots for distinct stages, ordered based on their accuracy for both injury and entrapment labels. Next, we identify the best algorithm for each stage.

As illustrated in Figure 4 and Figure 5, the majority of the algorithms demonstrated an increase in their accuracies after the implemented modifications, with some reaching near‐perfect performance. Additionally, it can be observed that the models of the modified stages exhibited quite similar performance to the final models when all data was available, indicating that the addition of new features enabled high accuracy at earlier stages. Table 7 and Table 8 present the most effective models along with their corresponding accuracy for every stage.

Figure 4.

Comparing the accuracy values of algorithms for firefighter injury.

Figure 5.

Comparing the accuracy values of algorithms for firefighter entrapment.

Table 7.

Identifying the best models for each stage for firefighter injury.

Stage	Best model	$\mathbf{Accuracy}\mathit{Avg}\mathit{\pm }\mathit{SD}\mathbf{Firefighters}\mathbf{’}\mathbf{Injury}$
T103	AutoMLP	$85.4\pm 1.5$
T104	AutoMLP	$96.9\pm 0.6$
T103M	MLP	$96.7\pm 0.7$
T104M	MLP	$97.6\pm 0.6$
T103–T107	MLP	$97.9\pm 0.3$

Table 8.

Identifying the best models for each stage for firefighter entrapment.

Stage	Best model	$\mathbf{Accuracy}\mathit{Avg}\mathit{\pm }\mathit{SD}\mathbf{Firefighters}\mathbf{’}\mathbf{confinement}$
T103	Kernel Naive Bayes	$91.6\pm 1.4$
T104	Kernel Naive Bayes	$93.1\pm 2.0$
T103M	Kernel Naive Bayes	$96.0\pm 1.6$
T104M	MLP	$97.7\pm 1.2$
T103–T107	AutoMLP	$98.3\pm 0.9$

According to Table 7 and Table 8, it is evident that the highest accuracy is achieved when all the data (collected data from stage T103 to T107) is available. However, using all data is not practical for firefighting organizations, as they need to identify potential injury and potential entrapment as early as possible. As shown Table 4, the models built for modified stages T103 and T104, exhibit comparable performance (in terms of accuracies) to the models that used all the data. Thus, T103M and T104M data are very useful for building predictive models for injury and entrapment, which addresses the needs of firefighting organizations.

There are other criteria for evaluating forecasting models, the list of criteria and the value obtained for each of the models are listed in Supporting Information S1: Table A2 and Supporting Information S1: Table A3, respectively.

4.9. Empirical Implementations

To implement the innovation presented in this article, a proposed infrastructure is detailed in the Supplemental Empirical Implementations section.

5. Discussion

In this study, we tested machine learning models to predict the occurrence of firefighter injury and entrapment in the early stages of urban fire situations. Firefighting operational data is acquired over time, and as the operation develops, more data is collected. As a result, the accuracy of the prediction models based on all of the data acquired about the operation was very high. On the other hand, because more data is acquired at the start of the operation, the accuracy of the models in the early stages of the process must be enhanced.

The study focused on two essential issues: (1) enhancing model accuracy early on (before dispatching the operational team) and (2) coping with unbalanced data. To solve the first difficulty, the researchers added additional features to the initial firefighting operation data and developed models to predict a huge amount of data collected in the latter stages of the operation before deploying the crew. The addition of more features considerably increased model accuracy, bringing it near to that of models trained using all available data. To solve the second difficulty, the researchers used oversampling techniques to balance class distributions while preserving all of the potentially meaningful observations.

The Multi‐Layer Perceptron and Kernel Naive Bayes models predicted firefighter injuries and entrapments with the highest accuracy, 96.7% and 96.0%, respectively. The data added in the T103M and T104M stages considerably increased the models' accuracy. According to [22, 23, 45], the type of accident site can have a substantial effect on enhancing the accuracy of fire prediction models. Initial discovery, main occupancy group, energy causing ignition, fuel or energy associated with igniting object, material first ignited group, igniting object group, and place are all critical factors in predicting firefighter injuries, according to the findings of [22].

Overall, the study's findings highlight the significance of applying machine learning algorithms to increase the effectiveness and safety of firefighting operations. Using these strategies in an emergency situation can assist firefighters lessen the probability of entrapment or injury while also improving emergency response time and outcomes. Firefighting companies must embrace predative modeling methodologies, notably the usage of machine learning algorithms, as a decision support system to optimize emergency response and preparedness.

Socioeconomic status [13, 46] and census‐level geographic region [1, 2, 47] have been utilized to predict their detrimental implications. The incorporation of features in this research data can help to increase model accuracy, and this study also employed the municipal region as a tool to create the prediction model. Incorporating physiological and occupational therapy techniques into injury and entrapment prediction models can enhance their accuracy by providing a more comprehensive understanding of the underlying biomechanical and functional factors involved. Physiological techniques, such as motion capture and electromyography, can provide detailed data on joint kinematics, muscle activation patterns, and loading conditions during various activities [48]. On the other hand, occupational therapy techniques focus on evaluating an individual's ability to perform daily activities and identifying potential barriers or risk factors within their environment [49]. By incorporating assessments of functional performance, environmental constraints, and activity demands, occupational therapy techniques can enhance the ecological validity of prediction models, ensuring they accurately reflect real‐world scenarios. With the features added into the models, we can adjust the granularity level of injury and entrapment prediction from the operation to the firefighter. The establishment of a platform that can convey this information to the FOMS could improve the capability of anticipating firefighter injuries as well as lowering the negative effects of fire occurrences.

In urban firefighting operations, the integration of advanced sensing devices, such as flame detectors and carbon dioxide (CO2) sensors, plays a crucial role in predicting firefighter injury and entrapment. By continuously monitoring these environmental parameters, these sensors provide real‐time data that can be used to assess the risk of injury and entrapment, allowing for timely interventions and strategic decision‐making during firefighting operations [50, 51, 52].

The use of a navigation platform, such as an autonomous ground wheeled robot, further enhances the safety and efficiency of urban firefighting operations. Equipped with a communication and localization network based on Bluetooth Low Energy (BLE) IoT technology, these robots can maintain reliable connectivity even in confined and obstructed spaces. The BLE IoT network enables precise indoor localization, helping to track the positions of firefighters and the robot itself. This integrated system allows for better coordination and situational awareness, reducing the risk of injury and entrapment by guiding firefighters to safe zones and exit points. The combination of IoT technology with advanced sensing and navigation platforms thus represents a significant advancement in enhancing the safety and effectiveness of urban firefighting operations [50, 53, 54, 55].

A DSS based on machine learning can be constructed based on the accuracy of the models. The benefit of a DSS in anticipating firefighter injuries lies from its capacity to provide real‐time monitoring and data analytics, which can improve the safety of firefighters. A DSS can forecast potential dangers and warn firefighters about unsafe conditions by evaluating past injury data. A DSS can also provide tailored suggestions by assessing the data of each fireman and providing particular recommendations based on their risk profiles. Using a DSS to anticipate firefighter injuries can assist in reducing injuries, preventing deaths, and making firefighting safer.

5.1. Limitations and Strength

The fact that this paper uses data from only one firefighter group in a single city is one of its limitations. It is likely that different organizations collect different data, making applying the model to data from other organizations challenging. Furthermore, the peculiarities of different cities, such as weather and building styles, may alter the chance of injury and entrapment. Another limitation of this article is that the complex machine learning models used in this research, such as deep learning, are often referred to as “black boxes” because their internal workings and decision‐making processes are opaque and difficult for humans to understand directly [56, 57]. These models learn intricate patterns from data in ways that are not easily interpretable, making it challenging to understand how they arrive at their predictions or decisions [57].

Despite the limitations noted above, this research has several strengths, most notably the earlier use of modified analysis to forecast the possibility of injury and entrapment. This information could considerably increase firefighters' readiness before responding to an incident. In addition, rather than relying on a single model, the inclusion of eight alternative models shows the robustness of the conclusions.

6. Conclusion

The primary goal of this study was to enhance the predictive capabilities of models used in the early stages of urban firefighting operations. By innovatively modifying two pre‐existing time stages and integrating additional, contextually relevant features, our approach has successfully demonstrated that machine learning models can provide reliable forecasts of firefighter injuries and entrapments, even with incomplete data. The use of Multi‐Layer Perceptron and Kernel Naive Bayes models, in particular, marked a significant step forward, showing the highest accuracy in predicting these critical outcomes.

This research underscores the potential of machine learning to aid decision‐makers in urban firefighting by providing early and actionable insights, thereby enhancing operational efficiency and reducing the impacts of urban fires. Moving forward, the inclusion of broader urban data sets—such as city infrastructure, population behavior, and building types—could further refine the predictive accuracy of these models.

Overall, our study's findings can help shape strategies and policies aimed at mitigating the negative consequences of urban fires and improving the safety and well‐being of both firefighters and the general public.

Author Contributions

Mohammad Mahdi Barati Jozan: conceptualization, methodology, formal analysis, visualization, investigation, writing–original draft. Hamed Khosravi: methodology, formal analysis, visualization, investigation, writing–original draft. Aynaz Lotfata: methodology, formal analysis, writing–review and editing. Krzysztof J. Cios: methodology, formal analysis, writing–review and editing. Hamed Tabesh: conceptualization, methodology, formal analysis, writing–review and editing; Supervision.

Ethics Statement

The ethics committee of Mashhad University of Medical Sciences approved the study (ID: IR. MUMS. MEDICAL. REC.1398.911). All participants provided written informed consent to participate in the study. All methods were carried out in accordance with relevant guidelines and regulations.

Conflicts of Interest

The authors declare no conflicts of interest.

Transparency Statement

The lead author Hamed Tabesh affirms that this manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned (and, if relevant, registered) have been explained.

Supporting information

Supporting information.

Data Availability Statement

The datasets used in this study are available upon request from the first author.