New approach for feature selection by using information entropy

Feature selection plays an important role in data mining and pattern recognition, and feature evaluation is the key issue for constructing a feature selection algorithm. In this work, we introduce information entropy to propose incomplete information entropy in incomplete information systems, and that in incomplete decision systems. Some important propositions and theorems for reducts are presented. Thus, we construct a forward greedy algorithm for feature selection. Theoretical analysis indicates that the new approach is suitable for not only both complete and incomplete information systems, but also both complete and incomplete decision systems, especially large ones. Furthermore, experimental results demonstrate that the proposed technique is effective and efficient. 1548-7741/Copyright © 2011 Binary Information Press.