Parallel Anchors Hierarchy to Document Signatures Using Sparse Spatial Selection

Abstract

Most of the automated documents clustering algorithms face the challenges of high cost and performance in comparing documents in large text corpora. Parallel anchors hierarchy is an algorithm that localizes data based on triangle inequality obeying distance metric, the algorithm strives to minimize the number of distance calculations needed to cluster the documents into “anchors†around reference documents called “pivots†and also improves the performance of clustering by adding parallelism to it. This algorithm selects the initial pivot at random and also decides the number of anchors to create before anchors formation. This may affect the clustering quality. In this paper, we extend the original parallel anchors hierarchy algorithm to improve the clustering quality and performance by adding dynamic pivot selection technique which decides the number of anchors to create based on the dimensionality of the text corpora.

 

Key words: parallel clustering, anchors hierarchy, Sparse spatial selection, pivot selection, database, documents.