history based heuristic feed querying scheme in web feed aggregator

Abstract

In the beginning of the web, the users used to visit a website to obtain information or content. However, a much more convenient communication model, known as content syndication, has gained a lot of attentions. Conventional web content syndication technologies like RSS and ATOM becomes popular from its simplicity. While Internet users don’t need to visit every site for the contents but only referring to feed reader, it appears that these content syndication technologies have a problem of inefficient feed query, resulting item missing and post-it delay. Moreover, feed query makes cost for the use of traffic. However, this problem has not been noted much and also few have attempted to address the problem. To solve this problem, history based pattern learning approaches are mainly used. It tracks publishing pattern and determines the number of queries which is used for feed querying and query timing schedule of those queries. Nevertheless, most studies only focus on the post-it delay for restricted query resource but it is not perfect for representing convenience in real use case. It needs to also consider possibility of item missing with small additional expense. In this paper, it presents the problem in detail, and proposes a new feed querying scheme which is used in web feed aggregator. It firstly aims to reduce item missing probability and then seizes chance of reducing post-it delay. Heuristic approach is adopted to handle various feed publishing pattern. To evaluate proposed scheme, it simulates the idea with feed data which is crawled from real feed publishers including popular news, magazines and various blogs. The simulation confirms that proposed scheme shows meaningful performance improvements in both item missing probability and post-it delay compared with previous studies.