Two Validated Ways of Improving the Ability of Decision-Making in Emergencies; Results from a Literature Review

Abstract

The aim of the current review wasto study the existing knowledge about decision-making and to identify and describe validated training tools.A comprehensive literature review was conducted by using the following keywords: decision-making, emergencies, disasters, crisis management, training, exercises, simulation, validated, real-time, command and control, communication, collaboration, and multi-disciplinary in combination or as an isolated word. Two validated training systems developed in Sweden, 3 level collaboration (3LC) and MacSim, were identified and studied in light of the literature review in order to identify how decision-making can be trained. The training models fulfilled six of the eight identified characteristics of training for decision-making.Based on the results, these training models contained methods suitable to train for decision-making.

Introduction

There are situations in which decisions must be made quickly, e.g., management of different types of crisis (manmade or natural) [1]. All decisions, including those made under pressure of time constraints, need a thorough consideration, since they might influence a whole chain of future events [2]. Thus, decision-making is seldom a simple act of choosing between two or more issues/situations.

In the crisis management context, decisions are deemed to be made by intuition and/or reasoning [3-5]. Based on Hassinet al. and Salas et al. [3, 6], two distinct information processing systems can be identified in the human brain: one conscious and deliberative and the other unconscious and intuitive. Salas [3] suggests, however, an integration of intuition and expertise, i.e., expertise-based intuition promoting rapid and accurate decisions.

Betsch[7] defines intuition as a process of thinking based on knowledge stored in the long-term memory. The intuition of an expert seems to develop with growing experience to a qualitative different intuition, a so-called educated intuition [8]. Betsch[7] describes the output process of intuition as a feeling that can serve as a basis for decisions, something fundamentally different from conscious and analytical reasoning.

Intuition-based decision-making is mostly based on our consciousness or “gut feeling” [3]. Avoiding unwanted outcomes of decisions needs a proper planning based on expectations and risk assessment, which in turn needs predictions of upcoming events. Although predictions of upcoming events, i.e., accidents, crisis, disasters, etc. might be difficult, we may stage and simulate various scenarios, to identify our shortcomings, strength and weaknesses in order to improve the decision-making. Unplanned and unexpected situations are often resolved by using this kind of decision-making, which also results in establishing risk guidelines to give a structure in approaching the different events [4, 5, 9]. As an effect, emergencies are often handled through step wise protocols, guidelines, and recommendations [10].

A combination of knowledge, experience, facts, and figures may be needed as the foundation for decision-making. These qualities and characteristics need to be improved by training in an environment that allows for repetitive decision-making and failure without harm [11, 12].

Although this can be accomplished by lectures and theoretical models, hands-on training in a close-to-real situation is ideal and also needed [11, 12]. There are models for making a proper, correct, but not necessarily quick decision, most of which give some structure in the process of decision-making. However, one major problem with these models is their inability to cover all the possible scenarios. Thus, there is a need for scenario-based training tools to expose decision-makers to different possibilities and situations [9-12].

Within the crisis and disaster management, the 4Cs [Command, Control, Communication, and Collaboration] are important key factors and involve the whole chain of actions in all managerial levels from the operational to the strategic leaderships [13, 14]. In every step and in certain sequences in real-time, decisions are made to be communicated for further implementation, and every decision leads to an action that must be conducted in collaboration with others. The outcome, if wrong, may have severe future consequences, e.g., death of innocent people.

Although education and training in proper situations can enhance the abilities and capabilities of 4Cs, it is clear that these capabilities cannot be tested during real incidents. Collaboration is often based on pre-disaster understanding, which is facilitated by planning and training [15]; moreover, training can induce mutual understanding and foster a common terminology [16]. Thus, an appropriate environment should be available where candidates can test their knowledge and gain skills and proficiency by putting such knowledge into practice. Repeated scenario-based training in decision-making for all managerial levels and in collaboration with all authorities is therefore needed [11, 12]. In this study, we decided to study two validated exercise systems, 3 level collaboration (3LC) and MacSim, in light of the literature review in order to identify how decision-making can be trained.

Material and Methods

A comprehensive literature review of articles published in PubMed, Google Scholar and electronic databases at public and university libraries was conducted by using the following keywords: decision-making, emergencies, disasters, crisis management, training, exercises, simulation, validated, real-time, command and control, communication, collaboration, and multi-disciplinary in combination or as an isolated word.

Results

 Publications

The search results were different for Pub Med and Google Scholar [see below]. The results obtained were matched after narrowing the outcome by adding a new keyword to the first search. Based on the list of keywords mentioned above, the final number reached was 291 publications for Google Scholar and 5 for Pub Med. Case studies and publications in non-scientific journals were excluded. The search in Google Scholar was limited to 2010–2016, due to a huge number of hits.

Google Scholar: Decision-making [1,520,000 hits], Emergencies [323,000 hits], Disasters [75,000 hits], Crisis management [16,500 hits], Training [16,200 hits], Simulation [11,000 hits], Validated [5,040 hits], Real-time [2,790 hits], Command and control [1,270 hits], Communication [1,250 hits], Collaboration [1,050], and Multi-disciplinary [219 hits]. Pub Med: Decision-making [228,760 hits], Emergencies [844 hits], Disasters [587 hits], Training [126 hits], and Simulation [4 hits].

Two validated exercise systems, 3 level collaboration (3LC) and MacSim, were studied in depth, in light of the literature review. The main reason for choosing these training methods was the fact that based on the authors’ experience and to the best of their knowledge, 3LC and MacSim were the only scientifically validated training methods in the available literature.

Contents

 Learning can be described in a simplified dichotomy of deep and superficial [17-21]. In the former, participants are considered to interact critically with the information presented, relate to their prior knowledge, and examine the logic of the arguments and the evidence that exists to draw conclusions. “Deep learning,” provides the participants with useful knowledge and lessons to be utilized at relevant actual events [18, 19]. The latter, “superficial learning,” on the other hand, means to memorize the contents of the information presented and to accept them without questioning [19, 20]. Although the dichotomy between deep and superficial learning has been problematized [20, 21], the main concern in the literature remains to be the choice of training, which can promote certain forms of learning for the benefit of the participants in an institutional level, dominated by customs, traditions, and automatic behaviors. Learning is considered to be a collective process, where the individual learns from each other, shares experiences, and develops common approaches, while challenging values and traditions [21-25].

Ford and Schmidt [26] define learning as a relatively permanent change in knowledge, skills, and attitudes. According to the authors, training can accomplish learning that includes the acquisition of knowledge, skills, and attitudes during the actual training. In contrast, inhibitors or obstacles in the transfer setting of the training can lead to a minimal impact of training on the individual's behaviors and performance on the task. This can be an effect of the fact that emergency response training has a tendency to have a narrow focus on the development of technical skills and expertise. 

Alliger et al., [27] suggest that the effect of training can be measured in the transfer of learning. The concept ‘transfer of training’ is based on the knowledge that learning from training has a tendency to be de-coupled from the actual crisis work. Three main reasons to such a de-coupling has been suggested: 1) design of the training, including the scenario; 2) teamwork during the training; and 3) the context where experiences from the training is supposed to be practiced [28, 29]. The primary goal for training is to influence the participant’s behavior at a real event and in the long-term [23-25]. In other words, the knowledge obtained from the training should be transferred to the real event. Thus, failure in this transformation process questions the effects of the training [30]. Habits and traditions may be obstacles for this transferring process [31]. Deep learning requires repeated interactions with others to discuss, simulate, and test new behavior patterns [32]. The close exchange between the participants promotes the learning [33, 34], which may have an effect on the participant’s collective action in a given scenario and last for a long time [21, 35].

Although training is viewed as a foundation for safer outcomes [36], there are only a few studies on its effects on professional emergency staff [37]. Such studies should take all elements of 4Cs into consideration and create alternative strategies, by which the participants may compare, discuss, and analyze the outcome of a training at a vertical level [within the organization] and at the horizontal level [in collaboration with other organizations] in complex situations [11, 12, 38-45]. In a multi-disciplinary situation, similar to emergency/disaster management, the majority of organizations prefer to work sequentially and in parallel [in turn and side-by-side] rather than synchronously [across boundaries]. Thus, even if, they may have a good outcome as a group, the lack of collaboration results not only in difficulties in communication [to understand each other's behavior, culture, rules, concepts, and symbols], but also in command and control and all hierarchical levels [11, 12, 46-49]. Failing social and organizational components can only be compensated, to a limited degree, by devices such as radio communication technology. The response system, according to Drabek [50], should be conceptualized as a socio-technical system, and the social or organizational component of such a system is more important than the technological component. With regard to command, control, and communication in emergencies, different types of simulations exercises in combination with didactic information has been identified as an effective choice of teaching, since it provides the possibility to immediately exercise and put into practice the gained knowledge. In addition, working in the simulated environment together with other team members may enhance performance, possibly help reduce errors, improve collaboration, and has an additive benefit to the traditional didactic instruction [11, 12, 51, 52]. Real-time scenarios seem to be the best choice for real time actions, since the consumed time in any measure conducted [e.g., putting an IV-line or the time it takes to transport, etc.], is taken into consideration. In such an exercise, the input data and the outcome are predictable and reflect the actual data/situation; thus, the outcome of the scenario can be evaluated. The knowledge and skills can also be practiced and learned without any hesitation, while the consequences of each decision made can be identified and discussed. Such a training can also be made feasible with regard to the time, duration, number of participants, and instructors [11, 12, 49, 51, 52].

Teamwork and collaboration promote deep learning and break the multi-disciplinary boundaries in emergency and crisis management. Simulation exercises offer a safe environment and possibilities for repetitive and fearless training and a final space for discussion and evaluation. The latter increases the reliability and the usefulness of the training and eases the multi-disciplinary cooperation. They also open up a new field for repetitive decision-making in a chain of commands on different managerial levels [11, 12].

There are eight relatively well documented aspects of how decision-making can be trained: 1) the importance of common seminars during training. In these seminars, participants inform each other, exchange experiences, and do common analysis of the decision-making. Such seminars are especially fruitful if they give room for reflection and reveal shortcomings [34, 39, 53-57]; 2) the advantage of comparing the collaborating organization’s rules, roles, and routines during the decision-making and making them transparent to the participants [58-60]; 3) focusing on collaboration to stimulate the practice of common decision-making across professions and organizational boundaries [61-63]; 4) the complexity and extensiveness of scenarios in decision-making (less complicated and less extensive scenarios will prepare participants for the most complicated and extensive event such as a disaster [64-66]; 5) the advantage of clarifying the role distribution between the participants. Decision-making will be beneficial if the roles of the participants are clarified [38, 56, 67-70]; 6) testing of different strategies of decision-making in a training improves the collaborative ways to make decisions during the actual crisis work [57, 69, 71, 72]; 7) identification of mistakes in the decision-making reduces mistakes and improves practice [24, 34, 37, 57, 73-76]; and 8) scenarios challenging the decision-making among and between the various organizations and professions stimulate the learning as well as increase the usefulness from the training [63].

In this review, two types of training models were found to be validated through multiple publications, dissertations, and years of experience: three level collaboration (3LC) and Mass Casualty Simulation (MacSim). Both are scenario-based, where the input and output data can be controlled, which also gives the possibility to adjust the severity of the exercise based on the participants´ experience and knowledge and can be used together to teach, exercise, and train different elements of the 4Cs.

3LC

3LC is a training model used to train small groups of commanders up to hundreds of participants in emergencies and major incidents. The development of the 3LC model was based on the hypothesis that the collaborative elements in a mutual task help to reduce the organizational barriers [18]. Organizational abilities and limitations were enlisted to promote an interplay with no hierarchal authority, as well as to promote the ability to switch between different collaboration strategies as demanded by the specific situation [53, 66, 77]. Collaboration training offers a chance to not only exhibit stability [the qualities that one develops through drill and practice], but also to practice transitions, overlaps, fearlessness, improvisation, creative thinking, and the ability to handle unexpected situations [78, 79]. Such learning is beyond the repeated learning that comes from the drill in control and command structures and other mechanistic structures. Moynihan [57] emphasizes the value in being able to try it out, i.e., to test a model to cope with an event, evaluate the effect of the model in the next stage, create a new approach, test it, and develop the method continuously.

Through the 3LC-exercise, the participants practice their ability to make a concerted and coherent assessment of which collaboration form is applicable at a given time. The collaboration is not a static activity, but subject to construction, deconstruction, and reconstruction, depending on what is appropriate for a specific time, place, or event. The exercise is called 3LC to illustrate the main goal of practicing the different forms of collaboration and the possibility of switching between them by including asymmetries in the scenarios [48, 80]. For example, the police can arrive first at the scene and begin the task of caring for many with serious injuries, or, understaffed emergency rescue services have to get help from the police and paramedics at a fire incident (Figure 1).

Fig. 1.

Starting picture from one of the 3LC table-top scenarios (Copyright 3LC, with permission).

The 3LC-exercises include three seminars and two exercise sessions. During the seminars, the following are discussed: (1) what has been done; (2) all the mistakes; (3) alternative strategies; (4) comparisons between the different strategies; and (5) suggestions for improvement [80]. An essential part of the exercises is to design them in such a way that staff from different organizations are invited to seamlessly overlap with each other. Seminar I aims to provide the participants with brief practical information regarding the exercise’s purpose. In Exercise I, the collaboration exercise is conducted either in table-top or full-scale format. The exercise is stopped when all the participants have repeated their efforts. In Seminar II, all those practicing together go through what had been carried out in the first practical exercise session in a single group. Exercise II is identical to the first. The staff meet in the same assembly area, are given the same information, and encounter a similar scenario, but with the exception that they can choose other strategies than those in Exercise I. Seminar III aims to see what was performed differently compared to the first time. The exercise leader continuously notes what improvements are made in Exercise II [63].

By using seminars, repetitions, and interactive documentation, 3LC offers all the participants an opportunity to obtain knowledge about each other's roles and understand the logic behind their actions, agendas, concepts, and hierarchical levels [37, 56, 81]. This model fulfills six of the documented aspects of training for decision-making (1,3,5,6,7,8, see above) and allows for an interaction in a safe environment and creates a functioning work flow and a joint decision-making as a basis for further development. Finally, the model contributes to trust building as a prerequisite for open communication and across-boundary work that facilitates the decision-making [31, 82-84].

MacSim

 MacSim is developed for scientific development and evaluation of methodology used in the medical response to major incidents and disasters. It can also be used for education and training for the medical response, from basic to advanced level and for staff of all categories. In addition, it can be used to test an existing organization with regard to planning and preparedness for, and performance in, major incidents and disasters. It is a validated training method that mobilizes and utilizes all available resources. It employs simplified up to more detailed methods of caring, diagnosing, and treating victims in an integrated multi-disciplinary alert- and response process and different levels of healthcare [11, 12].

The idea with MacSim is to intensively obtain the needed knowledge and skills required by interactive training, that is, “learning by doing.” Although field exercises can be preferable, they are too expensive, require many figurants, disturb the daily work, may only test part(s) of the chain of action, and are difficult to arrange. To overcome all these issues, MacSim uses real-time simulation. The exercises are run with either fictive resources in “Any Land,” where resources (devices, vehicles, staff, etc.), plans, and distances are already outlined, or with real resources in a specific country with regard to its devices, staff, and vehicles, etc. In the former, the available resources are released in a given timetable after the alert, while in the latter the distribution of resources follows the ordinary schedule of the country/region. Medical staff, rescue services, or police perform their tasks with access to the given or their real number of staff. A mass-casualty scenario is given and the scene is built up, together with a chain of pre-hospital transportation (ambulances and dispatch center). Hospitals and command centers are placed far from the scene, and all communication is conducted by using radio communication or telephones. The patient cards used in MacSim are authentic, taken from real incidents (Figure 2).    

Fig. 2.

Patient simulation used at MacSim simulation (copyright www.macsim.se, with permission).

MacSim allows for training and evaluation of the whole chain of response [scene, transport, hospitals, command, and communication] and its outcome. It makes it possible to train and evaluate coordination between the different components of this chain, which by experience is one of the most critical parts of the response and a frequent reason for failure [31].

The keystone of major incident response is decision-making, on all managerial levels, i.e., operational, tactical, and strategic levels, i.e., hands on work, optimal use of resources, and predictive plans. Based on the available information (input data), decisions are made, which consequently will affect the consumption of time, resources, and also the outcome of the patients (output data) and gives an opportunity to evaluate all the decisions made. This is possible since the system includes: 1) input data about the available resources, demographic and geographic conditions, and results of the patients’ examinations and 2) output data (mortality and morbidity related to injury severity, consumption, and utility of time and resources) [51, 52, 85].

The system includes common seminars where the participants exchange experiences and do common analyses of the decision-making. During the seminars, the routines used for the decision-making between the organizations and the professions are compared. Furthermore, the role distribution between the participants is clarified. The training involves testing of different scenarios and evaluating the outcome of different strategies. All participants are evaluated by pre- and post-course tests and face-to-face observations and evaluations by an instructor. The model fulfills six of the documented aspects of training for decision-making (1, 3, 5, 6, 7, 8 see above).

Discussion

Challenging decisions demand more considerations and can be affected by complexity and interpersonal issues [1-5, 9, 10]. This calls for guidelines and supporting documents. There are decision-making models targeting the financial and business world, e.g., Vroom-Yetton-Jago Decision model [86], most of which are not adjusted to the healthcare industry and are time consuming. Healthcare oriented decision-making tools are often used to engage patients in a shared decision with the physicians [87]. In routine healthcare situations, all decisions are ideally made based on reasoning, facts, and figures. In an emergency, guidelines and plans are developed to be used as a supportive tool for the decision-making, e.g., hospital´s disaster plans [88]. However, plans and guidelines have to be used, tested, and evaluated frequently to yield the desired result. Since disasters and major incidents are rare, there is a need for evaluable and validated training models.

All elements of 4Cs are important in managing emergencies and thus should be learnt and trained. As described in this paper, command, control, and communication are better learnt and trained in a simulation exercise targeting all the managerial levels and the whole chain of events, while collaboration is best learnt and trained by using a collaborative tool. As 4Cs are all integrated, the learning and training of all parts should be integrated or be conducted in close time connectedness. The best choice for learning and training is an evaluable and interactive method such as 3LC and MacSim that repeats the training sequences, evaluates the learning in seminars, and gives room for testing different strategies [11, 12, 63]. Although there are many different training methods, the importance, functionality, and efficacy still need to be studied systematically in order to validate the outcome [43, 69, 74, 89, 90].

In a recent study, it was reported that collaborative elements in a training contribute to perceived learning, and learning, in turn, had a perceived beneficial effect on the actual emergency work [91]. In other words, theoretically, a good collaboration results in a better outcome of an emergency work, which also embraces other parts of 4Cs, i.e., command, control, and communication. However, having a good command, control, and communication in a specific organization does not necessarily improve the collaboration ability. Training in these elements, thus, demands a new form of training or an integrated training, consisting of two or more training systems.

There are many challenges in creating a training program. One major challenge is to create a training that really results in deep learning, i.e., a collective process where the individuals learn from each other, share experiences, and develop common approaches, while challenging all the values and traditions [18, 21, 22, 41, 42, 78, 92, 93]. Accepting the fact that a training should result in long-term, deep learning, there should then be some demands on a selected system. For the benefit of all elements of 4Cs, scenario-based training is reported to be the best choice. However, the scenario should have some major characteristics. It must be so close to reality that participants experience uncertainty, frustration, and time constraints that they will encounter in a real incident [37, 54, 94, 95]. It must also challenge and encourage teamwork at all levels of 4Cs without being too exaggerated, unrealistic, or complex to create confusion [96]. Several studies have reported that confusion induces a reversion to overlearned (i.e., highly familiar) behavior and a focus on individual tasks that limit collaboration [25, 39, 40, 93, 97]. Although they may point out some shortcomings at best [capacity evaluation], overly complex and advanced exercise scenarios lack the ability to show how to manage these shortcomings and may lead to a confusion shared by all the participants [38, 98, 99].

A training should also create a reasonable balance between stability and change, i.e., the changes obtained or suggested as being needed after a training should not jeopardize the stability of the group examined [44]. High Reliability Organizations (HROs): police, ambulance, and rescue services are all regarded as stable, reliable, and predictable organizations [74, 100]. Such familiarity gives them confidence, as they know exactly what to do within their own organization. Any effort to jeopardize this familiarity by any kind of exercise requires practice drills and well-practiced routines [101]. Such effects can be obtained by using a simulation training, e.g., MacSim. In contrast, collaboration training such as 3LC requires the ability to adapt and change [90]. Standard operating procedures function as a stable base to perform repeated professional procedures. This base of pre-determined procedures can effectively be used in complex situations when improvisation and inter-organizational collaboration is needed [102]. A collaboration training is about being able to understand the situation and shift strategies depending on what the situation demands. It may require other strategies than those practiced, in order to tailor efforts and ensure a smooth normalization after an event [49, 66, 77, 103]. Finally, efficacy of a training should increase by performance in open forums, i.e., when participants from different organizations come together for discussion with no prestige [57] and provide a secure environment for reflection and enhancement of new ways of thinking and a safe evaluation [17, 53, 95].

Although collaboration exercises are relevant for crisis management, the existing research shows that they are exposed to several challenges [36, 79, 104, 105]. One problem is the difficulty to implement smooth transitions between organizations that effectively manage multiple organizations simultaneously [25, 38, 71, 106]. The need for a stable collaboration network has also been emphasized as an important factor [107]. Furthermore, it might be hard for involved agencies to understand each other’s capabilities and limitations, actions, logic, agendas, legislation, and hierarchical levels [37, 56, 81]. Finally, it has also proved difficult to learn from similar events from other organizations [108]. One reason for this may be that the behavior is deeply rooted and organizations may have difficulty in learning from past mistakes [34, 109]. Altogether, it is clear that for 4Cs training and evaluation, we need to have a set of training models that complete each other and cover all levels. MacSim, as a real-time simulation model, covers all those areas that 3LC, as a collaboration exercise may not cover. While 3LC prepares the participants to not only focus on their own tasks, but also being prepared to take spontaneous actions over the organizational borders [31], MacSim gives them the opportunity to train in identifying, recording, changing focus, and re-prioritizing resources depending on what the situation demands [74, 105, 110, 111].

One reason why critical incidents are not managed optimally is the lack of conditions for learning [112]. Such shortcomings seem to be handled by using 3LC and MacSim in combination [53]. Discussion and evaluation make participants critically interact and relate the content of the exercise to their own experience and thus lead to a deeper learning [17, 19, 53]. Another outcome is the trust which is built between all the actors and which is necessary in an environment without normative finger pointing, e.g., from exercise leaders who are unhappy with the participants’ contribution [113].

The 4Cs organizational belonging is both cultural and structural. The premise is that learning happens culturally rather than structurally. However, it should not create conflicts with all the structural elements of 4Cs. An organizational learning, based on the common experiences, norms, and perceptions, requires a high degree of involvement from the participants in a training [114]. In contrast, a structural approach to learning is based on the idea that formal rules, structure, and procedures contribute to efficiency. The fact that participants have the opportunity to highlight and discuss their own and the organizational weaknesses in both the training models is especially important for learning. The development of an open and unpretentious attitude in both models is another successful outcome [54, 56, 74, 80]. The opportunity to make mistakes, identify mistakes in a self-critical discussion, and then to be permitted to correct the weaknesses in a repetitive manner in 3LC and in another scenario in MacSim is important and beneficial for learning and increases the degree of trust and transparency [11, 12, 31, 80, 113].

According to Klabbers [33], repeated interactions contribute to deep learning that results in a long-term effect on behavior [16, 50]. The repetition phase of the training, i.e., the ability to “do it over – do it better,” creates reasonably large prerequisites for deep learning and a long-term impact [21, 35]. Together, they not only improve the collaboration needed for an interagency teamwork, but also command, control, and communication. Earlier reports suggest that 3LC contributes to learning and usefulness, as a high degree of collaboration in everyday work will increase the preparedness for unknown and difficult events. Chaotic situations with extensive resource asymmetry and lack of resources can supposedly be managed effectively if collaboration is applied repeatedly in the daily work. This requires good command, control, and communication not only in one’s own organization, but also between the different agencies. The experiences gained by using MacSim indicate that it improves the latter, as it allows for organizational improvement within and outside an agency [51, 52].

According to Bradley [115], the following features of high-fidelity simulation have an impact on learning: provision of feedback, repetitive practice, integration within a curriculum, provision of a range of difficulties, being adaptable [allowing multiple learning strategies and provision of a range of clinical scenarios], provision of a safe and educationally supportive learning environment, offering an active learning based on individual needs and defined outcomes. Both presented models in this paper, alone, or together, contain these features. In addition, MacSim uses different modules and provides multiagency cooperation and communication with available input and outcome data [51, 52], and the interagency collaboration is significant to 3LC. Simulation based training offers many advantages as an approach for management and education in not only medical fields, but also in other educational fields [11, 12].

This study emphasizes a need for validated training models for crisis and disaster management due to the fact that training models otherwise are based on experience and routine rather than scientifically evaluated impact. Non-validated models are considered to be effective in learning without being really scientifically evaluated. Since routine and daily decision-making in healthcare is based on facts and figures, they are obviously evaluable and thus should be examined based on evaluable methodology.

Author contributions:

Khorram-Manesh, A. has planned, reviewed, and wrote the manuscript. Berlin, J. and Carlström, E. have reviewed and critically examined the facts and improved the text. The final manuscript was approved by all authors.

Conflict of Interest:

None declared.