Word-IPCA: An Improvement in Dimension Reduction Techniques

Text Documents have high dimensions and processing knowledge from such huge magnitude of data is cumbersome. High dimensionality has inherent noise and sparsity. Dimension reduction is a preprocessing step in Text Mining which transforms data sets into more compact form and eliminates redundancy. Many statistical techniques like Principal Component Analysis (PCA), Independent Component Analysis (ICA) and their variants are available but these are incapable to check semantic information. Semantical techniques like Latent Semantic Analysis (LSA) help to find the semantic relationship but have computation limitation. This paper focuses on study and comparative analysis of dimension reduction techniques. The proposed hybrid method combines the success of statistical approach, semantic approach and then checks the extracted features with WordNet dictionary. This approach will help to reduce the dimensions and fulfill the purpose of semantic check.