Mining unstructured web pages to enhance web information retrieval

One major approach for information finding in the WWW is to navigate through some web directories and browse them for the goal pages. However, such directories are generally constructed manually and have disadvantages of narrow coverage and inconsistency. In this work, we propose a machine learning approach that automatically constructs a navigational structure for the WWW to help information finding. A self-organizing map is constructed to train the web pages and obtain two feature maps, which reveal the relationships among web pages and thematic keywords respectively. We then use these maps to develop a structure that may assist the users finding the information they need. We used a small set of web pages in the experiments and obtained promising result.