Incremental probabilistic learning of schema and case role assignment

Abstract

Case role assignment(CA), assigning proper thematic case roles to sentence constituents, is an important aspect of the natural language processing. Conventional approaches based on some ad-hoc scheme provide no schema learning procedure and degree of goodness of the CA decision. In this dissertation, a probabilistic model of case assignment is proposed, which is based on an incremental probabilistic learning of schema. The schema is a set of generalized case frames and captures probabilistic rules of alternative CA decisions with degrees of goodness of fit from training data. McClleland and Kawamoto``s, and Miikkulainen and Dyer``s PDP model also learn schema by the back-propagation learning procedure on a feedforward network and they applied the learned schema to the CA task. However the back-propagation learning requires time-consuming cyclic presentation of a complete set of examplars until the network weights converge to a common solution. Furthermore, it is non-incremental. In other words, if new examplars are to be adapted in the net, then all the prior examplars also should be presented cyclically. The output activation patterns of the models are sometimes unclear, so the interpretation of them demands quite a good deal of efforts. To overcome the weakness of the back-propagation learning procedure in CA task, a new model is proposed, which is based on the structure of a probabilistic network. In the model, nodes and links are dynamically created for each new feature-pattern in the schema. Then the probabilistic weights and thresholds of nodes are directly estimated by the new incremental probabilistic learning procedure that uses the observed frequency of training data. To prevent the explosion of nodes for featurepatterns, generalizations of the feature-patterns are carried out whenever a new full pattern is created so that nodes and links of subsumed patterns are dynamically destroyed. Application of the learned schema to the case assignment for input...