Sequential decision making problems for operation management of emergency medical services system in mass-casualty incidents

Abstract

In a mass-casualty incident (MCI) in which a large number of patients occur within a short time, since many casualties simultaneously require emergency medical services (EMSs), it is essential to optimize EMS resource operation management to minimize the damage to patients in the aftermath of an MCI. Therefore, this dissertation deals with the problems that the emergency medical service system should decide to provide optimal services to a large number of patients in an MCI. The major issues that the EMS system should consider in an MCI are the issues of prioritizing the transport of patients and the selection of destination hospitals. Patients in an MCI are classified into four classes (immediate, delayed, minor, expectant) by Simple Triage and Rapid Treatment (START) triage method. Of the classified patients, immediate class patients are transported first and then the emergency group of patients are transported. Minor class patients move to hospitals by self-transport and expectant class patients are transported last. Although there have been a number of studies on patient transport prioritization, such as suggesting that prioritizing immediate class patients prior to delayed class patients is not as effective at maximizing the number of survivors, there is little research on hospital selection. In Chapter 2, as the first decision problem, we solved the problem of determining which class of patients should be transported to which hospital for immediate class patients and delayed class patients. To solve this problem, this dissertation designed a Markov decision process (MDP) model and derived policy using reinforcement learning (RL). Experiment results using real accident data showed that making decisions by considering both decision-making problems are superior, and that selecting the appropriate hospital is as important as prioritizing patient transport. Furthermore, we proposed a heuristic algorithm that can easily and quickly determine the patient transport priority and the destination hospital in the accident site based on the hospital information. We also confirmed that the proposed algorithm had better performance than the other heuristic algorithms in literature. MDP used in the study of Chapter 2 cannot derive the optimal solution when the state space is large. In this case, RL is used to obtain a policy of near-optimal performance. However, when the state space of a model is very large, it is confirmed that it is difficult to derive a high-performance policy from existing RL algorithms. In Section 3, we proposed a meta-algorithm for the RL algorithm to achieve good performance for very large state space problems. The proposed meta-algorithm targets finite-horizon MDP. The main ideas of the proposed meta-algorithm are three-fold. First, we reduce the size of the original problem by dividing the state space of the original problem into two subspaces based on the flow of time. Second, we derive the policy for some problems in the second subspace (the latter part) and learn the obtained policy. Finally, we use the learned policy in the simulation to help solve the problem in the first subspace. Using the Temporal Difference learning and Deep Q-Network as examples of the RL algorithm, we showed that the meta-algorithm is able to reliably derive a high-quality policy solution. Through empirical analysis, we also found that a smaller second subspace in the meta-algorithm yields higher performance. In order to further improve the performance of the proposed meta-algorithm, we developed an extended version of the meta-algorithm using the concepts of iterative and ensemble methods. Another problem with the operation of the EMS system in an MCI is the problem caused by self-transport of minor class patients. Most patients in an MCI are minor class patients, and many of these patients go to the hospital themselves. Generally, they move to nearby hospitals or high-tier hospitals. However, if a large number of minor class patients are concentrated in nearby hospitals, there is a high likelihood of delay in treatment for serious patients requiring treatment at the hospitals. In Chapter 4, as the second decision problem, we solved the problem of assigning a hospital to minor class patients. In this case, the people who perform the decision are different from the person who makes the decision. The general optimization model assumes that people that perform the optimal decision are rational, but in reality, there are some cases where they do not follow optimal decisions depending on their mental inertia. Therefore, we proposed a new MDP modeling technique that considers the possibility that decision-performers make irrational decisions by their mental inertia without following given decisions. We derived the optimal policy from the original MDP model and the proposed model for the hospital assignment problem for minor class patients and compare them. Results showed that solving the problem by considering the behavior of the decision-performer leads to better results in reality. This dissertation proposes new algorithms and modeling techniques for sequential decision-making models as well as addressing various problems for effective operation management of EMS resources in an MCI. This study aims to contribute to the improvement of the ability to respond to MCIs from the viewpoint of the EMS system and to the development of research related to sequential decision-making model.