28 xnx2x1w2wnw11w0ΣfyFigure 12. A typical neuron model The output computation is done by mapping the weighted linear sum through an activation function: yf net=()There are two basic types of activation functions ± hard and soft. Hard activation function implies that the output of a neuron can exist in only one of the two possible states as shown below. ynetwnetnet=−=><sgn(),,01000Such neurons are generally called discrete neurons or perceptrons. Sometimes the two allowed states are 1 and -1. Such neurons are called bipolar discrete neurons. Neurons with soft activation functions are called softneurons or continuousneurons. Two types of soft activation functions are used ± sigmoidaland hyperbolic tangent. The sigmoidal activation function is given by ynetw=+−−110(exp(())αwhich produces a continuously varying output in the range [0 1]. The hyperbolic tangent function for activation yields a continuous output in the range [±1 1]. This function is given by ynetwnetw=−−−+−−(exp(())(exp(())1100αα
29 The quantityαin the above equations determines the slope of the activation function. Neural Network Models A neural network is a collection of interconnected neurons. Such interconnections could form a single layer or multiple layers. Furthermore, the interconnections could be unidirectional or bi-directional. The arrangement of neurons and their interconnections is called the architectureof the network. Different neural network models correspond to different architectures. Different neural network architectures use different learning procedures for finding the strengths (weights) of interconnections. Learning is performed using a set of training examples. When a training example specifies what output(s) should be produced for a given set of input values, the learning procedure is said to be a supervisedlearning procedure. This is the same as using a set of pre-classified examples in statistical data analysis and pattern recognition. In contrast, a network is said to be using an unsupervisedlearning procedure when a training example does not specify the output that should be produced by the network. While most neural network models rely on either a supervised or an unsupervised learning procedure, a few models use a combination of supervised and unsupervised learning. There are a large number of neural network models, as shown in Figure 13, which have been studied in the literature. Each model has its own strengths and weaknesses as well as a class of problems for which it is most suitable. We will briefly discuss only three models here that are common in data mining applications. These are (1) single-layer perceptron(SLP) network; (2) multiple-layer feed-forward network; and (3) self-organizing feature map. For information on other models, the reader should refer to books on neural networks.