Artificial neural networks (ANNs) are learning algorithms in the form of computer programs or hardware. ANNs are characterized by an architecture and a method of training. Network architecture refers to the way processing elements are connected and the direction of the signals exchanged. A processing element or unit is a node where input signals converge and are transformed to outputs via transfer or activation functions. The values of outputs are usually multiplied by weights before they reach another node. The purpose of training is to find optimal values of these weights according to a criterion. In supervised training, inputs are presented to the network and outputs are compared to the desired or target outputs. Weights are then adjusted to minimize an objective function such as the root mean square error for instance. In unsupervised training, the network itself finds its own optimal parameters.
Although there are several types of neural networks, a simple example of ANN is the multilayer perceptron. The middle sets of units are called hidden layers and the other two input and output layers. The transfer functions in the input and output layers can be identities, and those of the hidden layer are usually sigmoid or hyperbolic tangent functions. These functions map the sum of weighted inputs to the range between zero and one or between minus one and plus one. The flow of signals in the example is unidirectional giving the name feed forward to the whole network. One can have also the output from the network and connect it to the inputs thus leading to recurrent networks which are useful for time series modeling. Typically, the hidden layers contain several processing elements. Obviously the outputs are modeled as highly non-linear functions of the original inputs. Thus, it is the architecture of units that allow an ANN to be a universal approximator. In other words an ANN can recover an unknown mapping from the input to the output space as long as it contains enough processing elements (White et al., 1992). The network can be trained with back propagation (Rumelhart and McClelland, 1986), which seeks a minimum in the error function via the gradient descent method. Weights are adjusted in the direction that reduces the value of the error function after each presentation of the input records.
ANNs sometimes share the problem of local minima and the problem of overtraining. Because of the non-linearity involved, the algorithm may not always reach a global minimum. Overtraining refers to the situation where the network literally memorizes the inputs and cannot generalize (predict well) when it is applied to a new set of data. However, there are ways to overcome these problems and ANNs are very useful. In fact on many occasions they are superior to linear models in terms of prediction accuracy. A correctly trained network should be able to generalize, that is, to recognize patterns in data it has not yet seen. Although statistical measures such as ^-ratios are not available, one can perform sensitivity analysis. This consists of varying one input within a reasonable range and observing how the estimated output function behaves.
Neural networks have been successfully applied in finance and economics, although research in this area is still new. Examples include forecasting security prices, rating bonds, predicting failure of banks or corporate mergers, and conducting portfolio management (Refenes, 1995). Although statistical models and ANNs overlap considerably, the two sets of models are not identical. White (1989) and Kuan and White (1992) discuss the parallels between statistical or econometric models and feedforward networks. Cheng and Titterington (1994) study ANNs from a statistical perspective, and Ripley (1994) compares standard classification techniques with ANNs. Classification is an area in which neural networks have been useful because they are often capable of sharply discriminating between classes of inputs with different characteristics. The general literature on ANNs is extensive. Hecht-Nielsen (1990) and Wasserman (1993) are two introductory topics. The Internet news group comp.ai.neural_nets is an informative forum for exploring this growing field.
