DISTRIBUTED ALGORITHM FOR CLUSTERING LARGE DATASETS

Abstract

In this paper , we propose a novel scheme called Prefix-Suffix trees which is a compact and complete representation of the database that requires only a single database scan for its construction. We propose a clustering algorithm based on this scheme and a parallel implementation of the algorithm based on the message passing model. The proposed algorithm exploits the inherent parallelism in the algorithm. The algorithm is implemented on a cluster computer with 3 nodes running LINUX. We compare the execution times experimentally and show that the parallel algorithm is efficient than its sequential counterpart.