Digital neighbourhoods: Partitioning the Web for information indexing and searching

General purpose centralized whole Web indexes have become increasingly harder to set up and maintain with the scaling problem of Web size. In this article, we present a novel approach to information indexing and searching based on partitioning the Web according to Digital Neighbourhoods (DNs). We first introduce the concept of DN and we present four definitions which allow for the characterization of indexing and searching scopes. We then present the architecture of a DN-based distributed system and its characteristics of effective indexing, scalability and interoperability. Finally we describe the structure of a working prototype and some implementation details.