Probabilistic parsing of Korean based on language-specific properties

Abstract

Natural language parsing is a central component to many natural language processing tasks. Since a natural language has inherently structural ambiguities, one of the difficulties of parsing is resolving the structural ambiguities. The ambiguities arise whenever a sentence can be interpreted in more than one way. Recently, a probabilistic approach to tackle this disambiguation problem has received considerable attention because it has some attractions such as automatic learning, wide-coverage, and robustness. Many probabilistic parsing models have been developed for the past few years, and they are mainly for English not Korean. In this thesis, we focus on Korean syntax and a probabilistic parsing model for Korean. We investigate two problems: representing Korean syntax, and building a language model for Korean syntax. The representation of a language is to describe the structure of the language, and it is the direct reflection of the features of the language. A language model is a probability distribution $P(S)$ over string $S$ that attempts to reflect how frequently a string $S$ occurs as a sentence. The claim of this thesis is that syntax of Korean in fact can be represented more efficiently using a grammar representation scheme that exploits the characteristics of Korean. Also, we argue that the consideration of the language specific features can produce a more accurate natural language parser than one without such consideration. These claims are justified by constructing a parser for Korean based on the specific properties of the Korean structures, and comparing its performance to a state-of-the-art parser for English on a common task.