Intelligent Web mining model to enhance knowledge discovery on the Web

The large size and the dynamic nature of the Web highlight the need for continuous support and updating of Web based information retrieval systems. Crawlers facilitate the process by following the hyperlinks in Web pages to automatically download a partial snapshot of the Web. This paper describes some details about the architecture of a fully implemented a Multi-Agent Web Search System I-Spider for the Internet. Its architecture is based on autonomous software agents and the paper is focused on the communication among them. The overall system architecture is based on a multi-agent paradigm. Agents collaborate together HTML pages from the World Wide Web and treat them in order to be able to retrieve those pages from subsequent users' queries. Crawling Agent collaboration is required in order to decide the URLs that should be first retrieved Subsequent page treatment consists on first filtering the pages so that HTML format is transformed into XML and second indexing them so that information retrieval can be performed online.