p-SARS, Peer-to-peer Search for A Recommender System

15.12.2003

Abstract

WAIF started out in 2002 and the overall goal is to make the computers automatically search for relevant information based on the user’s preferences, and to push this information directly to the user wherever he/she is and to whatever device he/she has available. In other words, make the machines serve us, with as little human interaction as possible. This thesis focuses on a specific part of this problem, which is to design and implement a search mechanism and the surrounding distributed system that can find publishers that publish on a given topic. We present a search mechanism that is scalable, fault resistant, self administrative and that utilizes the resources already present in the network. This is done by utilizing the powers of the unstructured overlay peer-to-peer architecture.

However creating an efficient search mechanism for a pure peer-to-peer net is known to be a problem due to the decentralized nature of these overlay networks. Our solution is to incorporate several known techniques. We propose the use of Random Walks supplemented by both a hint cache and a probabilistic gossiping mechanism. The results gathered show that the search mechanism has good coverage but is highly dependent on that the time to live (TTL) set on the query reflects the size of the overlay network and that the nodes individual hint caches are populated.

To verify our design we have both implemented the system and a simulator. We show with throughput testes and simulations that the system designed can scale to millions of users.


Keywords:
Peer-to-peer, unstructured, search, gossip, Information Retrieval (IR)