You will now be shown how the applet is able to recall a pattern. This is done by
presenting the input sequence to the Neural Network and determining the output for each Neuron. The mathematical basis for this process was already described in the section "Recalling Patterns." The application determines the output of the Neural Network by using the Layer class, shown in Listing 2.2.
Listing 2.2: The Layer Class (Layer.java) public class Layer {
/**
* An array of neurons. */
protected Neuron neuron[] = new Neuron[4]; /**
* The output of the neurons. */
protected boolean output[] = new boolean[4]; /**
* The number of neurons in this layer. And because this is a * single layer neural network, this is also the number of * neurons in the network.
*/
protected int neurons; /**
* A constant to multiply against the threshold function. * This is not used, and is set to 1.
*/
public static final double lambda = 1.0; /**
* The constructor. The weight matrix for the * neurons must be passed in. Because this is * a single layer network the weight array should * always be perfectly square(i.e. 4x4). These * weights are used to initialize the neurons. *
* @param weights A 2d array that contains the weights between each * neuron and the other neurons.
*/
Layer(int weights[][]) {
neurons = weights[0].length; neuron = new Neuron[neurons]; output = new boolean[neurons]; for ( int i=0;i<neurons;i++ )
neuron[i]=new Neuron(weights[i]); }
/**
* The threshold method is used to determine if the neural * network will fire for a given pattern. This threshold * uses the hyperbolic tangent (tanh).
*
* @param k The product of the neuron weights and the input * pattern.
* @return Whether to fire or not to fire. */
public boolean threshold(int k) {
double kk = k * lambda; double a = Math.exp( kk ); double b = Math.exp( -kk ); double tanh = (a-b)/(a+b); return(tanh>=0);
} /**
* This method is called to actually run the neural network. *
* @param pattern The input pattern to present to the neural network.
*/
void activation(boolean pattern[]) { int i,j; for ( i=0;i<4;i++ ) { neuron[i].activation = neuron[i].act(pattern); output[i] = threshold(neuron[i].activation); } } }
When the constructor is called for the Layer class, the weight matrix is passed in. This will allow the Layer class to determine what the output should be for a given input pattern. To determine the output sequence, an input sequence should be passed to the activation method of the Layer class. The activation method calls the each of the neurons to determine their output. The following code does this.
for ( i=0;i<4;i++ ) {
neuron[i].activation = neuron[i].act(pattern); output[i] = threshold(neuron[i].activation); }
The above loop stores each neuron’s activation value in the same neuron. Each activation is determined by calling the act method of the Neuron. To see how each neuron calculates its activation we must first examine the Neuron class, which is shown in Listing 2.3.
Listing 2.3: The Neuron Class (Neuron.java) public class Neuron {
/**
* The weights between this neuron and the other neurons on * the layer.
*/
public int weightv[]; /**
* Activation results for this neuron. */
public int activation; /**
* The constructor. The weights between this neuron and * every other neuron(including itself) is passed in as * an array. Usually the weight between this neuron and * itself is zero.
*
* @param in The weight vector. */
public Neuron(int in[]) {
weightv = in; }
/**
* This method is called to determine if the neuron would * activate, or fire.
*
* @param x Neuron input
* @return If the neuron would activate, or fire */
public int act(boolean x[] ) { int i; int a=0; for ( i=0;i<x.length;i++ ) if ( x[i] ) a+=weightv[i]; return a; } }
To calculate a neuron’s activation, that neuron simply sums all of its weight values. Only those weight values that have a 1 in the input pattern are calculated. This is accomplished by using the following code.
for ( i=0;i<x.length;i++ ) if ( x[i] )
a+=weightv[i]; return a;
Chapter 2: Understanding Neural Networks
Article Title: Chapter 2: Understanding Neural Networks Category: Artificial Intelligence Most Popular From Series: Programming Neural Networks in Java
Posted: Wednesday, November 16, 2005 05:14 PM Author: JeffHeaton
Page: 7/7
Summary
Neural Networks are one of the most commonly used systems in Artificial Intelligence. Neural Networks are particularly adept at recognizing patterns. This allows them to recognize something even when distorted.
A Neural Network may have input, output and hidden layers. The input and output layers are the only required layers. The input and output layer may be the same neurons. Neural networks are typically presented input patterns that will produce some output pattern. If a Neural Network mimics the input pattern it was presented with, then that network is said to be autoassociative. For example, if a neural network were presented with the pattern “0110”, and the output were also “0110”, then that network would be said to be autoassociative.
A neural network calculates its output based in the input pattern and the neural network’s internal connection weight matrix. The values for these connection weights will determine the output from the neural network, based upon input pattern.
A Hopfield neural network is a fully connected autoassociative neural network. What this means is that each neuron is connected to every other neuron in a Hopfield Neural Network. A Hopfield Neural Network can be trained to recognize certain patterns. Training a Hopfield Neural Network involves performing some basic matrix manipulations in the input pattern that is to be recognized.
This chapter showed how to construct a simple Hopfield Neural Network. The next chapter will show how to create a multi-layered neural network. To do this you will be introduced to the JOONE package that is freely available for Java.
Chapter 3: Using Multilayer Neural Networks
Article Title: Chapter 3: Using Multilayer Neural Networks Category: Artificial Intelligence Most Popular
From Series: Programming Neural Networks in Java
Posted: Wednesday, November 16, 2005 05:15 PM Author: JeffHeaton
Page: 1/5
Introduction
In this chapter you will see how to use the feed-forward multilayer neural network. This neural network architecture has become the mainstay of modern neural network
programming. In this chapter you will be shown two ways that you can implement such a neural network.
We will begin by examining an open source neural network engine called JOONE. JOONE can be downloaded from http://joone.sourceforge.net. JOONE contains a neural network editor that allows you to quickly model and test neural networks. We will use this editor to introduce the concept of how a multi-layer network fits together.
JOONE also includes an engine which can be used directly in any Java project. We will not be using the engine in this book. While the engine is quite capable, the focus of this book is more on how to implement a neural network. The JOONE Engine will be discussed later in this chapter. For now, we begin by examining the feed forward backpropagation neural network using the JOONE Editor.
Chapter 3: Using Multilayer Neural Networks
Article Title: Chapter 3: Using Multilayer Neural Networks Category: Artificial Intelligence Most Popular
From Series: Programming Neural Networks in Java
Posted: Wednesday, November 16, 2005 05:15 PM Author: JeffHeaton
Page: 2/5