Parallelization of Multi-query Processing for Hierarchical Data Streams

Abstract

Recently, as increasing amounts of information are stored, exchanged, and presented using eXtensible Markup Language (XML), it becomes more and more important to adequately process XML streams. Meanwhile, the multicore architecture has been the norm for all computing systems in recent years as it provides the CPU-level support of parallelism. However, existing algorithms for processing XML streams do not fully take advantage of the facility since they have not been devised to run in parallel. They also show a degraded processing performance as the number of user queries increases. In this thesis, we propose several methods to parallelize the finite state automata(FSA)-based XML stream processing technique efficiently. We transform a large collection of XPath expressions into multiple FSA-based query indexes and then process XML streams in parallel by virtue of index-level parallelism. Each core works only with its own query index so that no synchronization issue occurs while filtering XML streams with multiple path patterns given by users. Moreover, proposed algorithm permits query processing to share input scans and path solutions to reduce redundant processing and save computations and I/Os. We also present an in-memory MapReduce model that enables to process a large collection of twig pattern joins over XML streams simultaneously. Twig pattern joins in our approach are performed by multiple H/W threads in a shared and balanced way. In addition, we address performance issues in the in-memory MapReduce by providing a sophisticated run-time workload balancing scheme. It is achieved by computing the cost of each twig pattern join operation before actual joining. Extensive experiments show that our algorithm outperforms conventional algorithms by up to ten times on an 8-core CPU for processing 10 million XPath expressions over XML streams. Through extensive experiments with synthetic XML dataset, we prove that our parallel algorithms are efficient and scalable.