Scene text recognition using part-based character models

Abstract

Understanding scene images has attracted considerable attentions, and there have been many researches to solve the problem in the form of subproblems such as object detection, object recognition, and scene segmentation. Text in scene images is one of the most informative contents to understand the images. Scene text recognition is the problem of recognizing text in scene images taken in unconstrained manner. Many researches on scene text recognition have been proposed, but most of them utilize character models only in character recognition phase, the last stage of the process. In former phases such as text detection and text extraction, only abstracted features of text regions are used, which might cause loss of information. In this thesis, we propose a novel scene text recognition method which fully utilizes concrete models of target characters from the beginning to the end of the recognition process. Each of the target character set is modeled with a part-based object model called implicit shape model (ISM) to achieve robustness for partial degradation of characters. Towards this end, we trained a Hough forest which localizes character parts and casts probabilistic votes on possible positions of characters. The votes are aggregated in voting spaces via generalized Hough transform, and then character candidates are detected at the local maxima of the voting space. The detected character candidates are verified by organizing the most plausible text lines in a semi-Markov conditional random field (semi-CRF) framework where the optimal configuration can be efficiently found using dynamic programming. As concrete character models are utilized throughout the process, even extremely deformed text are detected and recognized, which are hardly detected with previous approaches.