Show description

When this situation occurs, a common approach to divide the available data into three disjoint sets, through the use of random selection, is shown in Fig. 11. 3. Unfortunately, the use of the terms “test set” and “validation set” are reversed between the statistics and machine-learning communities. This discussion will use the definitions common to the statistics community. 10 Randomly sampling population to form three independent data sets for independent-sample testing. Supervised Training Methods 45 model selection, and the test set is used to determine the generalization error of the final chosen model.

How the designer sets up the training data, along with the corresponding network topology, will definitely affect the results. This leads to some rules of thumb to use while training feedforward neural networks. 1 Foley’s Rule The ratio of the number of samples per class (S) to the number of features (N) should exceed 3 to obtain optimal performance (S/N > 3). Foley showed in his seminal paper [Foley, 1972] that when S/N > 3, the training-set error would approximate the test-set error and that the resultant error would be close to that of a Bayes optimal classifier.

1 Process of thresholding neural-network outputs to determine class membership. 34 Chapter 6 accomplish this, as represented by Eq. 1), where y represents the output of the neuron and y represents the rescaled output. If the target values were not initially scaled by the neuron’s activation function, as shown in Eq. 1), and the activation function was a sigmoid, the designer can include an output transform by an inverse sigmoid. This is shown in Eq. 2). 1) y = (Max_value − Min_value) ×f −1 y − Min_target + Min_value.