The neural network we presented in Chapter 1 is an example of a Hopfield network with a single layer. Now we present a C++ implementation of this network. Suppose we place four neurons, all connected to one another on this layer, as shown in Figure 4.1. Some of these connections have a positive weight and the rest have a negative weight. You may recall from the earlier presentation of this example, that we used two input patterns to determine the weight matrix. The network recalls them when the inputs are presented to the network, one at a time. These inputs are binary and orthogonal so that their stable recall is assured. Each component of a binary input pattern is either a 0 or a 1. Two vectors are orthogonal when their dot product—the sum of the products of their corresponding components—is zero. An example of a binary input pattern is 1 0 1 0 0. An example of a pair of orthogonal vectors is (0, 1, 0, 0, 1) and (1, 0, 0, 1, 0). An example of a pair of vectors that are not orthogonal is (0, 1, 0, 0, 1) and (1, 1, 0, 1, 0). These last two vectors have a dot product of 1, different from 0.
Figure 4.1 Layout of a Hopfield Network
The two patterns we want the network to have stable recall for are A = (1, 0, 1, 0) and B = (0, 1, 0, 1). The weight matrix W is given as follows:
C++ Neural Networks and Fuzzy Logic:Const1ructing a Neural Network
0 -3 3 -3 W = -3 0 -3 3 3 -3 0 -3 -3 3 -3 0
NOTE: The positive links (values with positive signs) tend to encourage agreement in a
stable configuration, whereas negative links (values with negative signs) tend to discourage agreement in a stable configuration.
We need a threshold function also, and we define it using a threshold value, [theta], as follows:
1 if t >= [theta]
f(t) =
{
0 if t < [theta]
The threshold value [theta] is used as a cut-off value for the activation of a neuron to enable it to fire. The activation should equal or exceed the threshold value for the neuron to fire, meaning to have output 1. For our Hopfield network, [theta] is taken as 0. There are four neurons in the only layer in this network. The first node’s output is the output of the
threshold function. The argument for the threshold function is the activation of the node.
And the activation of the node is the dot product of the input vector and the first column of the weight matrix. So if the input vector is A, the dot product becomes 3, and f(3) = 1. And the dot products of the second, third, and fourth nodes become –6, 3, and –6, respectively. The corresponding outputs therefore are 0, 1, and 0. This means that the output of the network is the vector (1, 0, 1, 0), which is the same as the input pattern. Therefore, the network has recalled the pattern as presented. When B is presented, the dot product obtained at the first node is –6 and the output is 0. The activations of all the four nodes together with the threshold function give (0, 1, 0, 1) as output from the network, which means that the network recalled B as well. The weight matrix worked well with both input patterns, and we do not need to modify it.
Classes in C++ Implementation
In our C++ implementation of this network, there are the following classes: a network class, and a neuron class. In our implementation, we create the network with four neurons, and these four neurons are all connected to one another. A neuron is not self-connected,
C++ Neural Networks and Fuzzy Logic:Const1ructing a Neural Network
though. That is, there is no edge in the directed graph representing the network, where the edge is from one node to itself. But for simplicity, we could pretend that such a connection exists carrying a weight of 0, so that the weight matrix has 0’s in its principal diagonal.
The functions that determine the neuron activations and the network output are declared public. Therefore they are visible and accessible without restriction. The activations of the neurons are calculated with functions defined in the neuron class. When there are more than one layer in a neural network, the outputs of neurons in one layer become the inputs for neurons in the next layer. In order to facilitate passing the outputs from one layer as inputs to another layer, our C++ implementations compute the neuron outputs in the
network class. For this reason the threshold function is made a member of the network
class. We do this for the Hopfield network as well. To see if the network has achieved correct recall, you make comparisons between the presented pattern and the network output, component by component.
Previous Table of Contents Next Copyright © IDG Books Worldwide, Inc.
C++ Neural Networks and Fuzzy Logic:Const1ructing a Neural Network
C++ Neural Networks and Fuzzy Logic
by Valluru B. Rao
M&T Books, IDG Books Worldwide, Inc. ISBN: 1558515526 Pub Date: 06/01/95
Previous Table of Contents Next
C++ Program for a Hopfield Network
For convenience every C++ program has two components: One is the header file with all of the class declarations and lists of include library files; the other is the source file that includes the header file and the detailed descriptions of the member functions of the classes declared in the header file. You also put the function main in the source file. Most of the computations are done by class member functions, when class objects are created in the function main, and calls are made to the appropriate functions. The header file has an .h (or .hpp) extension, as you know, and the source file has a .cpp extension, to indicate that it is a C++ code file. It is possible to have the contents of the header file written at the beginning of the .cpp file and work with one file only, but separating the declarations and implementations into two files allows you to change the implementation of a class(.cpp) without changing the interface to the class (.h).
Header File for C++ Program for Hopfield Network
Listing 4.1 contains Hop.h, the header file for the C++ program for the Hopfield network. The include files listed in it are the stdio.h, iostream.h, and math.h. The iostream.h file contains the declarations and details of the C++ streams for input and output. A network class and a neuron class, are declared in Hop.h. The data members and member functions are declared within each class, and their accessibility is specified by the keywords protected or public.
Listing 4.1 Header file for C++ program for Hopfield network. //Hop.h V. Rao, H. Rao
//Single layer Hopfield Network with 4 neurons #include <stdio.h> #include <iostream.h> #include <math.h> class neuron { protected: int activation;
friend class network; public:
int weightv[4]; neuron() {}; neuron(int *j) ;
C++ Neural Networks and Fuzzy Logic:Const1ructing a Neural Network
int act(int, int*); }; class network { public: neuron nrn[4]; int output[4]; int threshld(int) ; void activation(int j[4]); network(int*,int*,int*,int*); };
Notes on the Header File Hop.h
Notice that the data item activation in the neuron class is declared as protected. In order to make the member
activation of the neuron class accessible to the network class, the network is declared a friend class in the
class neuron. Also, there are two constructors for the class neuron. One of them creates the object neuron without initializing any data members. The other creates the object neuron and initializes the connection weights.
Source Code for the Hopfield Network
Listing 4.2 contains the source code for the C++ program for a Hopfield network in the file Hop.cpp. The member functions of the classes declared in Hop.h are implemented here. The function main contains the input patterns, values to initialize the weight matrix, and calls to the constructor of network class and other member functions of the network class.
Listing 4.2 Source code for C++ program for Hopfield network. //Hop.cpp V. Rao, H. Rao
//Single layer Hopfield Network with 4 neurons #include "hop.h" neuron::neuron(int *j) { int i; for(i=0;i<4;i++) { weightv[i]= *(j+i); } }
int neuron::act(int m, int *x) {
int i; int a=0;
C++ Neural Networks and Fuzzy Logic:Const1ructing a Neural Network for(i=0;i<m;i++) { a += x[i]*weightv[i]; } return a; } int network::threshld(int k) { if(k>=0) return (1); else return (0); }
network::network(int a[4],int b[4],int c[4],int d[4]) { nrn[0] = neuron(a) ; nrn[1] = neuron(b) ; nrn[2] = neuron(c) ; nrn[3] = neuron(d) ; }
void network::activation(int *patrn) { int i,j; for(i=0;i<4;i++) { for(j=0;j<4;j++) { cout<<"\n nrn["<<i<<"].weightv["<<j<<"] is " <<nrn[i].weightv[j]; } nrn[i].activation = nrn[i].act(4,patrn); cout<<"\nactivation is "<<nrn[i].activation; output[i]=threshld(nrn[i].activation);
cout<<"\noutput value is "<<output[i]<<"\n"; }
}
void main () {
int patrn1[]= {1,0,1,0},i; int wt1[]= {0,-3,3,-3}; int wt2[]= {-3,0,-3,3}; int wt3[]= {3,-3,0,-3}; int wt4[]= {-3,3,-3,0};
C++ Neural Networks and Fuzzy Logic:Const1ructing a Neural Network
cout<<"\nTHIS PROGRAM IS FOR A HOPFIELD NETWORK WITH A SINGLE LAYER OF"; cout<<"\n4 FULLY INTERCONNECTED NEURONS. THE NETWORK SHOULD RECALL THE"; cout<<"\nPATTERNS 1010 AND 0101 CORRECTLY.\n";
//create the network by calling its constructor.
// the constructor calls neuron constructor as many times as the number of // neurons in the network.
network h1(wt1,wt2,wt3,wt4);
//present a pattern to the network and get the activations of the neurons h1.activation(patrn1);
//check if the pattern given is correctly recalled and give message for(i=0;i<4;i++)
{
if (h1.output[i] == patrn1[i])
cout<<"\n pattern= "<<patrn1[i]<<
" output = "<<h1.output[i]<<" component matches"; else
cout<<"\n pattern= "<<patrn1[i]<< " output = "<<h1.output[i]<< " discrepancy occurred"; } cout<<"\n\n"; int patrn2[]= {0,1,0,1}; h1.activation(patrn2); for(i=0;i<4;i++) { if (h1.output[i] == patrn2[i])
cout<<"\n pattern= "<<patrn2[i]<<
" output = "<<h1.output[i]<<" component matches"; else
cout<<"\n pattern= "<<patrn2[i]<< " output = "<<h1.output[i]<< " discrepancy occurred"; }
}
Previous Table of Contents Next
Copyright © IDG Books Worldwide, Inc.
C++ Neural Networks and Fuzzy Logic:Const1ructing a Neural Network
C++ Neural Networks and Fuzzy Logic
by Valluru B. Rao
M&T Books, IDG Books Worldwide, Inc. ISBN: 1558515526 Pub Date: 06/01/95
Previous Table of Contents Next
Comments on the C++ Program for Hopfield Network
Note the use of the output stream operator cout<< to output text strings or numerical output. C++ has
istream and ostream classes from which the iostream class is derived. The standard input and output
streams are cin and cout, respectively, used, correspondingly, with the operators >> and <<. Use of cout for the output stream is much simpler than the use of the C function printf. As you can see, there is no formatting suggested for output. However, there is a provision that allows you to format the output, while using cout.
Also note the way comments are introduced in the program. The line with comments should start with a double slash //. Unlike C, the comment does not have to end with a double slash. Of course, if the
comments extend to subsequent lines, each such line should have a double slash at the start. You can still use the pair, /* at the beginning with */ at the end of lines of comments, as you do in C. If the comment continues through many lines, the C facility will be handier to delimit the comments.
The neurons in the network are members of the network class and are identified by the abbreviation nrn. The two patterns, 1010 and 0101, are presented to the network one at a time in the program.
Output from the C++ Program for Hopfield Network
The output from this program is as follows and is self-explanatory. When you run this program, you’re likely to see a lot of output whiz by, so in order to leisurely look at the output, use redirection. Type Hop
> filename, and your output will be stored in a file, which you can edit with any text editor or list by
using the type filename | more command.
THIS PROGRAM IS FOR A HOPFIELD NETWORK WITH A SINGLE LAYER OF 4 FULLY INTERCONNECTED NEURONS. THE NETWORK SHOULD RECALL THE PATTERNS 1010 AND 0101 CORRECTLY. nrn[0].weightv[0] is 0 nrn[0].weightv[1] is -3 nrn[0].weightv[2] is 3 nrn[0].weightv[3] is -3 activation is 3 output value is 1 file:///H:/edonkey/docs/c/(ebook-pdf)%20-%20mathem..._Neural_Networks_and_Fuzzy_Logic/ch04/058-061.html (1 of 3) [21/11/02 21:57:00]
C++ Neural Networks and Fuzzy Logic:Const1ructing a Neural Network nrn[1].weightv[0] is -3 nrn[1].weightv[1] is 0 nrn[1].weightv[2] is -3 nrn[1].weightv[3] is 3 activation is -6 output value is 0 nrn[2].weightv[0] is 3 nrn[2].weightv[1] is -3 nrn[2].weightv[2] is 0 nrn[2].weightv[3] is -3 activation is 3 output value is 1 nrn[3].weightv[0] is -3 nrn[3].weightv[1] is 3 nrn[3].weightv[2] is -3 nrn[3].weightv[3] is 0 activation is -6 output value is 0
pattern= 1 output = 1 component matches pattern= 0 output = 0 component matches pattern= 1 output = 1 component matches pattern= 0 output = 0 component matches nrn[0].weightv[0] is 0 nrn[0].weightv[1] is -3 nrn[0].weightv[2] is 3 nrn[0].weightv[3] is -3 activation is -6 output value is 0 nrn[1].weightv[0] is -3 nrn[1].weightv[1] is 0 nrn[1].weightv[2] is -3 nrn[1].weightv[3] is 3 activation is 3 output value is 1 nrn[2].weightv[0] is 3 nrn[2].weightv[1] is -3 nrn[2].weightv[2] is 0 nrn[2].weightv[3] is -3 activation is -6 output value is 0 nrn[3].weightv[0] is -3 file:///H:/edonkey/docs/c/(ebook-pdf)%20-%20mathem..._Neural_Networks_and_Fuzzy_Logic/ch04/058-061.html (2 of 3) [21/11/02 21:57:00]
C++ Neural Networks and Fuzzy Logic:Const1ructing a Neural Network nrn[3].weightv[1] is 3 nrn[3].weightv[2] is -3 nrn[3].weightv[3] is 0 activation is 3 output value is 1
pattern= 0 output = 0 component matches pattern= 1 output = 1 component matches pattern= 0 output = 0 component matches pattern= 1 output = 1 component matches
Further Comments on the Program and Its Output
Let us recall our previous discussion of this example in Chapter 1. What does the network give as output if we present a pattern different from both A and B? If C = (0, 1, 0, 0) is the input pattern, the activation (dot products) would be –3, 0, –3, 3 making the outputs (next state) of the neurons 0,1,0,1, so that B would be recalled. This is quite interesting, because if we intended to input B, and we made a slight error and ended up presenting C instead, the network would recall B. You can run the program by changing the pattern to 0, 1, 0, 0 and compiling again, to see that the B pattern is recalled.
Another element about the example in Chapter 1 is that the weight matrix W is not the only weight matrix that would enable the network to recall the patterns A and B correctly. If we replace the 3 and –3 in the matrix with 2 and –2, respectively, the resulting matrix would facilitate the same performance from the network. One way for you to check this is to change the wt1, wt2, wt3, wt4 given in the program
accordingly, and compile and run the program again. The reason why both of the weight matrices work is that they are closely related. In fact, one is a scalar (constant) multiple of the other, that is, if you multiply each element in the matrix by the same scalar, namely 2/3, you get the corresponding matrix in cases where 3 and –3 are replaced with 2 and –2, respectively.
Previous Table of Contents Next
Copyright © IDG Books Worldwide, Inc.
C++ Neural Networks and Fuzzy Logic:Const1ructing a Neural Network
C++ Neural Networks and Fuzzy Logic
by Valluru B. Rao
M&T Books, IDG Books Worldwide, Inc.
ISBN: 1558515526 Pub Date: 06/01/95
Previous Table of Contents Next