Semantic Analysis of Big Data by Applying De-duplication techniques

As the data available on the web is in heterogeneous formats such as text, video, audio etc. Hence, there is need to integrate the data from the different sources and analyze the data which can be utilized for efficient query execution. After integration the data becomes large and there is need to used different techniques to analyze data. Thus, a de-duplication technique has been introduced by integrating heterogeneous data in same format. Further, to analyze large amount of data, record level data duplication is being applied on integrated data that is also used for removing similar type of data. Finally, the experimental results validate the efficiency in terms of execution time, storage space and success.