Managing metadata for distributed information servers

In this paper we present the design and implementation of a distributed index architecture to facilitate metadata discovery on large networks. Many current information discovery systems use crawlers to search the Internet, constructing metadata from the data they find. Instead, we propose a general distributed index architecture that uses descriptive hierarchies to organize metadata and route metadata queries to sites likely to produce relevant results. We are testing the architecture in the NASA Earth Observing System domain. We address some requirements particular to EOS, then discuss the design implications. We also present the results of preliminary performance studies on a prototype.