Efficient signature file methods for information retrieval

Abstract

With rapid increase of information requirements from various application areas, there has been much research on the efficient information retrieval. An approach widely advocated for the efficient information retrieval is to use signature file method. The signature file is an abstraction of information and has been applied in many proposals of information retrieval systems. Many signature file methods have been developed to efficiently access unformatted data with low storage overhead. Some of them are for static environments and others are for dynamic environments. In general, however, all the previous methods have various problems such as inability to support range queries, lack of handling synonyms and so on. In addition, the static two-path methods that achieve good retrieval performance over other static methods suffer from path selection overhead. The previous dynamic methods also have the problem of serious performance degradation for light query signature weights. In this dissertation we investigate various signature-based file organizations that work efficiently in various information retrieval applications. We first propose a new hybrid access method that employs signature clustering. Our proposed hybrid method overcomes various problems such as inability to support range queries, path selection overhead and lack of handling synonyms. A clustering method that has been extensively examined in the literature of library science is promising one to deal efficiently with signature files. However, works on clustering methods for clustering document signatures have not been made in the past. We develop two new heuristic clustering methods for signature files, called CBS and CWD. The CBS and CWD alleviate the problems of the previous clustering methods by grouping document signatures instead of documents. The clustering effects of CBS and CWD are tested by using 20,000 real library documents. We apply CBS and CWD to our hybrid method to improve its retrieval p...