Evaluation of neural network performance and generalisation using thresholding functions

The application of a simple thresholding technique to help assess the satisfactory performance of classification networks formed from Multi-Layer Perceptron (MLP) artificial neural networks (ANNs) is discussed. Both conventional Maximum Likelihood and Bayesian Evidence based training paradigms were
implemented. Firstly a simulated data set drawn from a two-dimensional Gaussian distribution was investigated to illustrate the physical significance of the threshold plots compared to the classifier output probability contours. Secondly a real world application data set comprising of low-frequency vibration measurements on an aircraft wing (a GNAT trainer) is considered. It is demonstrated that simple threshold based plots applied to classifier network outputs may provide a simple yet powerful technique to aid in the rejection of poorly regularised network structures.