Semantic Fusion of Live Web Content: System Design and Implementation Experiences

Conventional Web search models are ineffective at providing quick and comprehensive answers to questions related to live content such as real-time data or temporal relationships between actors. Semantic data fusion techniques have the potential to provide a more suitable abstraction model for efficient search on this type of data. However, myriad architectural and technical implementation challenges arise when trying to implement a working system. This paper summarizes our efforts and experiences at implementing a functional semantic fusion system for live content from the Web. Besides semantic data fusion techniques, we make extensive use of natural language processing, semantic Web technologies and Bayesian statistics to render the system a self-contained framework acting directly between Web resources of interest and end-user search applications. We first present the semantic fusion architecture design that we have developed. We have implemented this architecture and tested its effectiveness using real-world live data from the Web over multiple weeks. We then report about our major experiences and lessons-learned of this experiment.