A framework for data mining and information retrieval on the Internet

This paper presents a new framework, called NetMiner, for data mining and information retrieval on the Internet. Combining data mining techniques with the Internet information resource, NetMiner aims to provide a powerful information mining engine. NetMiner can not only use keywords to retrieve information on the Internet in the same way as other search engines do, but is also capable of processing heterogeneous information retrieval requests, helping users extract high level information and making predictions on likely future outcome. The paper first discusses briefly some of the common information retrieval techniques currently used on the Internet. It then outlines our proposal for NetMiner for mining the Internet. An overall structure of NetMiner is described, together with discussion of various components of NetMiner and their interactions. The functionality of several key modules and some system issues are also highlighted.