Feature construction and feature selection in presence of attribute interactions

When used for data reduction, feature selection may successfully identify and discard irrelevant attributes, and yet fail to improve learning accuracy because regularities in the concept are still opaque to the learner. In that case, it is necessary to highlight regularities by constructing new characteristics that abstract the relations among attributes. This paper highlights the importance of feature construction when attribute interaction is the main source of learning difficulty and the underlying target concept is hard to discover by a learner using only primitive attributes. An empirical study centered on predictive accuracy shows that feature construction significantly outperforms feature selection because, even when done perfectly, detection of interacting attributes does not sufficiently facilitates discovering the target concept.