Neural Networks

Neural networks are used extensively in the business world as predictive models. In particular, the financial services industry widely uses neural networks to model fraud in credit cards and monetary transactions.

Neural networks attempt to mimic a neuron in a human brain, with each link described as a processing element (PE). Neural networks learn from experience and are useful in detecting unknown relationships between a-set of input data and an outcome. Like other approaches, neural networks detect patterns in data, generalize relationships found in the data, and predict outcomes. Neural networks have been especially noted for their ability to predict complex processes.

A processing element, or PE, processes data by summarizing and transforming it using a series of mathematical functions. One PE is limited in ability, but when connected to form a system, the neurons or PEs create an intelligent model. PEs are interconnected in any number of ways and they can be retrained over several, hundreds, or thousands of iterations to more closely fit the data they are trying to model.

Processing elements, or PEs, are linked to inputs and outputs. The process of training the network involves modifying the strength, or weight, of the connections from the inputs to the output. Increases or decreases in the strength of a connection are based on its importance for producing the proper outcome. A connection's strength depends on a weight it receives during a trial-and-error process. This process uses a mathematical method for adjusting the weights, and is called a learning rule.

Training repeatedly, or iteratively, exposes a neural network to examples of historical data. PEs summarize and transform data, and the connections between PEs receive different weights. That is, a network tries various formulas for predicting the output variable for each example.

Training continues until a neural network produces outcome values that match the known outcome values within a specified accuracy level, or until it satisfies some other stopping criterion.

Each of the processing units takes many inputs and generates an output that is a nonlinear function of the weighted sum of the inputs. The weights assigned to each of the inputs are obtained during a training process (often back-propagation) in which outputs generated by the net are compared with target outputs. The answers you want the network to produce are compared with generated outputs, and the deviation between them is used as feedback to adjust the weights. (Groth, 1999) The process of readjusting weights is important to increasing a model's accuracy. The number of hidden nodes can be adjusted, and, in fact, there can be multiple levels of hidden nodes, just to confuse matters. The number of inputs, hidden nodes, outputs, and the weighting algorithms for the connections between nodes determine the complexity of a neural network. In general there is a trade-off between the complexity of a neural network, its accuracy, and the time it takes to create the neural network model. Because the configuration of hidden nodes and weights is so critical

to neural networks, there are many approaches for finding the right number of hidden nodes and readjusting weights.

This is at best an introductory view of neural networks, but it does give a starting point from which to understand how they work.

The greatest strength of neural networks is their ability to accurately predict outcomes of complex problems. Neural networks are a preferred technique in performing estimation, or continuous numeric outputs, which are popular in financial markets and manufacturing. (Groth, 1999)

There are some downfalls to neural networks. First, they have been criticized as being useful for prediction, but not always in understanding a model. It is true that early implementations of neural networks were criticized as "black box" prediction engines; however, with the new tools on the market today, this particular criticism is debatable.

Secondly, neural networks are also susceptible to over-training. If a network with a large capacity for learning is trained using too few data examples to support that capacity, the network first sets about learning the general trends of the data. This is desirable, but then the network continues to learn very specific features of the training data, which is usually undesirable. Such networks are said to have memorized their training data, and lack the ability to generalize. Commercial-grade neural networks today have effectively eliminated overtraining through bootstrapping holdout (test) samples, and by monitoring test versus training errors. Over-training can be measured by periodically checking the results of your test data set. Early stages of a training session yield lower error measurements on both the training and the test data. This continues unless the network capacity is larger than need be, or unless there are too few data sets in the training file. If at some point during learning, your test data begin to produce worse results, even though the training data continue to produce improved results, over-training is occurring.

Another issue with neural networks is training speed. Neural networks require many passes to build. This means that creating the most accurate models can be very time consuming. It is only fair to mention that all regression techniques require time to converge; and, while back propagation is slow, training neural networks can be sped up dramatically with methods like conjugate gradient.