A fast Branch-and-Bound algorithm for U-curve feature selection

Abstract

We introduce a fast Branch-and-Bound algorithm for optimal feature selection based on a U-curve assumption for the cost function. The U-curve assumption, which is based on the peaking phenomenon of the classification error, postulates that the cost over the chains of the Boolean lattice that represents the search space describes a U-shaped curve. The proposed algorithm is an improvement over the original algorithm for U-curve feature selection introduced recently. Extensive simulation experiments are carried out to assess the performance of the proposed algorithm (IUBB), comparing it to the original algorithm (UBB), as well as exhaustive search and Generalized Sequential Forward Search. The results show that the IUBB algorithm makes fewer evaluations and achieves better solutions under a fixed computational budget. We also show that the IUBB algorithm is robust with respect to violations of the U-curve assumption. We investigate the application of the IUBB algorithm in the design of imaging W-operators and in classification feature selection, using the average mean conditional entropy (MCE) as the cost function for the search.