In this paper, we propose a max-margin learning algorithm of hidden Markov model for speech emotion recognition. A max-margin learning leads to a good generalization ability on testing data even with small number of training data which may lead to an over-fitting. In the experiment, we observed that the proposed learning algorithm outperforms the learning criteria such as the maximum likelihood and maximum mutual information.