This paper deals with the optimal decision problem in various dynamic, nonlinear, and stochastic operation management problems. In particular, sequential decision making is modeled by the Markov Decision Process, and we discuss how to solve it in each problem. This study covers the following four issues. 1) Optimizing Multistage University Admission Decision Process. 2) An Efficient Approximate Solution for Stochastic Lanchester Models. 3) The Optimal Fire Allocation Strategy in Network Centric Warfare. 4) Learning Job Dispatching using Supervised Learning. Each study has its own characteristics, and most of the applied fields are different, but commonly the modeling and the decision problems are solved, new modeling proposed for each problem, and new decision-making solutions have better performance than existing methods .