Back-Propagation (BP) Neural Network

Recurrent neural network (RNN) has been widely used as a tool in the data classification. This network can be educated with gradient descent back propagation. However, traditional training algorithms have some drawbacks such as slow speed of convergence being not definite to find the global minimum of the error function since gradient descent may get stuck in local minima. As a solution, nature inspired metaheuristic algorithms provide derivative-free solution to optimize complex problems. This paper proposes a new metaheuristic search algorithm called Cuckoo Search (CS) based on Cuckoo bird’s behavior to train Elman recurrent network (ERN) and back propagation Elman recurrent network (BPERN) in achieving fast convergence rate and to avoid local minima problem. The proposed CSERN and CSBPERN algorithms are compared with artificial bee colony using BP algorithm and other hybrid variants algorithms. Specifically, some selected benchmark classification problems are used. The simulation results show that the computational efficiency of ERN and BPERN training process is highly enhanced when coupled with the proposed hybrid method.

This work presents the implementation of trainable Artificial Neural Network (ANN) chip, which can be trained to implement certain functions. Usually training of neural networks is done off-line using software tools in the computer system. The neural networks trained off-line are fixed and lack the flexibility of getting trained during usage. In order to overcome this disadvantage, training algorithm can implemented on-chip with the neural network. In this work back propagation algorithm is implemented in its gradient descent form, to train the neural network to function as basic digital gates and also for image compression. The working of back propagation algorithm to train ANN for basic gates and image compression is verified with intensive MATLAB simulations. In order to implement the hardware, verilog coding is done for ANN and training algorithm. The functionality of the verilog RTL is verified by simulations using ModelSim XE III 6.2c simulator tool. The verilog code is synthesized using Xilinx ISE 10.1 tool to get the netlist of ANN and training algorithm. Finally the netlist was mapped to FPGA and the hardware functionality was verified using Xilinx Chipscope Pro Analyzer 10.1 tool. Thus the concept of neural network chip that is trainable on-line is successfully implemented.

the physicians. For each task of the decisions propagation algorithm is significant. Neural networks have recently attracted more attention due to their ability to learn complex and non-linear functions. Artificial neural networks can be viewed as parallel and distributed processing systems that consists of a large number of simple and massively connected processors. These networks can be trained offline for determining various faults in complicated mapping, and can be used in an efficient way in the online environment. Feed-forward neural networks are considered where back-propagation algorithm is applied and trains the given thyroid dataset. By training the given thyroid dataset we can classify the diagnosis of thyroid disorders. Back propagation algorithm has been applied to many pattern recognition problems. The neural network architecture have a common property that all neurons in a layer are connected to all neurons in adjacent layers through unidirectional branches. BP algorithm provides best results for this problem. In this paper, MATLAB is used for simulations

In this paper, two layer neural network back propagation method was proposed to diagnose the breast cancer. In two layer neural network back propagation algorithm input layer is not counted because it serves only to pass the input values to the next layer. Neural network is a set of connected input and output units in which each connection has weights associated with it. During the learning phase, the network learns by adjusting the weights. It has longer training time therefore suitable for many applications. The rest of the paper is organized as follows. In the second part of this study we discussed related work and the Data set. Section 3 describes the proposed work. Section 4 presents the Materials and method. Section 5 presents the results of our classification. Section 6 concludes the paper with discussion and some directions for future research.

In this work, a total of 322 tests were taken on young volunteers by performing 10 different falls, 6 different Activities of Daily Living (ADL) and 7 Dynamic Gait Index (DGI) tests using a custom-de- signed Wireless Gait Analysis Sensor (WGAS). In order to perform automatic fall detection, we used Back Propagation Artificial Neural Network (BP-ANN) and Support Vector Machine (SVM) based on the 6 features extracted from the raw data. The WGAS, which includes a tri-axial accele- rometer, 2 gyroscopes, and a MSP430 microcontroller, is worn by the subjects at either T4 (at back) or as a belt-clip in front of the waist during the various tests. The raw data is wirelessly transmit- ted from the WGAS to a near-by PC for real-time fall classification. The BP ANN is optimized by vary- ing the training, testing and validation data sets and training the network with different learning schemes. SVM is optimized by using three different kernels and selecting the kernel for best classi- fication rate. The overall accuracy of BP ANN is obtained as 98.20% with LM and RPROP training from the T4 data, while from the data taken at the belt, we achieved 98.70% with LM and SCG learning. The overall accuracy using SVM was 98.80% and 98.71% with RBF kernel from the T4 and belt position data, respectively.

The result showed the performance of neural network with resilient back propagation training method, support vector machine and radial bases function Neural Net- work for classifying of mental tasks w.r.t baseline. RBF (Radial Basis Function) neural network method has best performance among all the classifiers for classification of mental tasks w.r.t baseline. By using RBF Neural Network 100% accuracy was obtained. While classifica- tion, Resilient Back Propagation training method showed better performance than other (Gradient Descent method, Levenberg-Marquardt, Conjugate Gradient De- scent and Gradient Descent Back Propagation with movementum) back propagation training methods. The main conclusion is that the Radial baisis function net- work was found to be most suitable in various applica- tions of BCI systems.

Abstract: Rainfall is considered as one of the major components of the weather forecasting. In the current world climate change, the accuracy of rainfall forecasting model is very important factor. Rainfall affects the drought and flood situation. India is an agricultural country. Rainfall are also affects the area of agriculture. We have considered the monthly rainfall data of East Madhya Pradesh, India from 1901-2017. Afterwards, in this paper, to evaluate an actual prediction of rainfall forecasting, were used an Artificial Neural Network (ANN) by the Cascade-forward Back propagation Neural Network (CFBPNN) technique. In this study, to trained the training data of the rainfall information using 2 hidden layers of CFBPNN technique, with three different epochs: [2-50-10-1] with epoch fix to 500, [2-50-20-1] with epochs fix to 1000 and 1500. To measure the performance of the developed architecture, the mean square error (MSE) algorithm is employed using CFBPNN. The experimental results showed that [2-50-10-1] architecture with epoch fix to 500 and learning rate 0.1, produced a good performance result with the value of MSE was 0.0063408. Eventually, CFBPNN algorithm has provided a best accuracy model to predict monthly rainfall in East Madhya Pradesh, India.

Abstact: As the number and diversity of distributed information sources on the Internet exponentially increase, various search services are developed to help the users to locate relevant information. But they still exist some drawbacks such as the difficulty of mathematically modeling retrieval process, the lack of adaptivity and the indiscrimination of search. This paper shows how heteroge- neous neural networks can be used in the design of an intelligent distributed in- formation retrieval (DIR) system. In particular, three typical neural network models - Kohoren’s SOFM Network, Hopfield Network, and Feed Forward Network with Back Propagation algorithm are introduced to overcome the above drawbacks in current research of DIR by using their unique properties. This preliminary investigation suggests that Neural Networks are useful tools for intelligent search for distributed information sources.

outward toward centre of the retina. These features are used to train the Gaussian Support Vector Machine to label individual patches of image. Vijayakumari and SuriyaNaraynan [9] used template matching for optic disc detection. Authors used sobel edge detector to find objects with sharp edges and used enhanced MDD classifier to find yellowish objects. Asha gowda et.al [10] attempted a method in which Back propagation Neural Networks are used for exudates detection. Features like Hue, Intensity, Standard deviation of intensity, distance between mean of optic disc pixels and pixels of exudates and non-exudates and mean intensity have been used as inputs to train the neural network .Ivo soares et.al [11] used combination of morphological operators and adaptive thresholding. The authors claim that the method is unaffected by contrast changes, non-uniform illumination and variable background resulting in correct detection of exudates.

Birendra Kumar Patel[9] proposed an image compression technique which utilizes the back propagation neural network and also combining the Levenberg-Marquardt concept with it. The first step is to divide the input image into various sub-blocks and every sub- block is then passed to the network based upon the complexness of the value of sub-block. The Levenberg-Marquardt algorithm in combination with the back propagation neural network showed that compression of image and convergence time can be bettered. An adaptive approach has been proposed by Prema Karthikeyan[10] by changing the training algorithm of the back propagation network with Levenberg-Marquart method. Modified Levenberg-Marquart concept is used for selecting images that have to be compressed. This approach uses different networks for different input image blocks according to their complexity values. The experimental results show that a better quality of image is received by overlapping neighboring image blocks.

Prediction of a child HIV status poses real challenges in medical research. Even though there are different statistical techniques and machine learning algo- rithms that have been used to predict models like HIV for the clinical data with binary outcome variables, yet neural network techniques are major participants for prediction purposes. HIV is the primary cause of mortality among women of reproductive age globally and is a key contributor to maternal, infant and child morbidity and mortality. In this paper, resilient back propagation algo- rithm is used for training the Neural Network and Multilayer Feed forward network to predict the mother to child transmission of HIV status.

The dimensionality of input feature vector is changed by considering the small eigenvalues as zeros. The accu- racy of the obtained results depends upon the selected threshold value of an eigenvalue. If the threshold value of eigenvalue is high, then the dimension of feature ei- genvector is reduced more and hence the high compres- sion can be achieved. This method considers only the data which has the maximum variation. Thus, PCA gives the simple solution to achieve high data compression, image compression and eigenface recognition. But, at high threshold eigenvalue, the loss of image data is very large. To overcome this drawback in PCA, authors sug- gested a bottleneck type neural network which is used to change the dimension of feature eigenvector matrix effi- ciently as well as reconstruction of an original image data. The architecture of such bottleneck type feed forward back propagation neural network for compression appli- cation is shown in Figure 1.

Abstract: The concept of Intrusion Detection System is used in the work. The data set is used for training and testing. Various numeric features of dataset are selected for better accuracy.SVM that is Support Vector Machine is trained for classifying normal and intruded sessions in the dataset. The work is tested in various parameters like Accuracy, Recall, precision and F measure. Once the Intruded sessions are found, EBPNN that is Error Back Propagation Neural network is Trained and Tested for the type of intrusion they are DOS, R2l, U2R and Probe. The Accuracy is tested in second module also on the Basis of Recall, Precision, and F measure

Software Defect Prediction (SDP) focuses on the detection of system modules such as files, methods, classes, components and so on which could potentially consist of a great amount of errors. SDP models refer to those that attempt to anticipate possible defects through test data. A relation is present among software metrics and the error disposition of the software. To resolve issues of classification, for the past many years, Neural Networks (NN) have been in use. The efficacy of such networks rely on the pattern of hidden layers as well as in the computation of the weights which link various nodes. Structural optimization is performed in order to increase the quality of the network frameworks, in two separate cases: The first is the typically utilized approximation error for the present data, and the second is the capacity of the network to absorb various issues of a general class of issues in a rapid manner along with excellent precision. The notion of Back Propagation (BP) is quite elementary; the result of neural networks is tested against the desired outcome. Genetic algorithms (GA) are a type of search algorithms built, based on the idea of natural evolution. A neural network using Shuffled Frog Algorithm for improving SDP is proposed. Keywords: Software Defect Prediction (SDP), Neural Network (NN), Back Propagation (BP), Genetic

A Feed-Forward network consists of a series of layers. The first layer has a connection from the network input. Each subsequent layer has a connection from the previous layer. The final layer produces the network’s output. Feed- forward networks can be used for any kind of input to output mapping. Specialized versions of the feed-forward network include fitting (fitnet) and pattern recognition (patternnet) networks. Feed-forward backpropagation network is simply the application of backpropagation procedure into the feed-forward networks such that every time the output vector is presented, it is compared with the desired value and the error is computed. The error value tells us how far the network is from the desired value for a particular input and the backpropagation procedure is to minimize the sum of error for all the training samples.

Fig 4: Back Propagation Neural Network extended gradient descent based Delta learning rule, commonly known as Back Propagation rule. In order to train a neural network to perform some task, we must adjust the weights of each unit in such a way that the error between the desired output and the actual output is reduced. This process requires that the neural network compute the error derivative of the weights. In other words, it must calculate how the error changes as each weight is increased or decreased slightly. The back propagation algorithm is the most widely used method for determining the derivative of weights. The back propagation algorithm is easiest to understand if all the units in the network are linear. In this network, error signal between desired output and actual output is being propagated in backward direction from output to hidden layer and then to input layer in order to train the network.

Table 2 give an idea about the CPU time, number of epochs and the mean square error for the 2 bit XOR data sets with ten hidden neurons. From the table, we can identify that the proposed CSBP method has better result than the ABC-BP, ABC-LM and BPNN algorithm. The CSBP achieves a MSE of 0 in 100 epochs and in 28.9 second of CPU time. Meanwhile, the other algorithms have short performed with large MSE’s and CPU times. The Figure 2 shows the convergence performance of CSBP algorithm for the 2-10-1 network architecture.

Abstract: In modern world, medical data became more sensitive where the need of privacy became most prominent. In order to provide the privacy and authentication to the medical data which includes the medical records of a patient digital water marking technique is chosen. The aim of the paper is to authenticate the medical data by digital watermarking technique using neural networks. The host data is processed through 4 level DWT then applied to a back propagation neural network where the secret image is embedded along with different noise attacks. The obtained image is processed through IDWT which is either stored or transmitted. At viewer or receiver side the reverse process is done to extract the secret image from the watermarked image. Compared to existing neural network algorithms back propagation algorithm yields better PSNR values.

Using the Back-Propagation Network, characters drawn with the help of a stylus can thus be recognized. The algorithms used are shown and the results using them are given. The results of the shown back-propagation approach are acceptable for recognizing characters. The time taken for training is a factor and for large number of characters, the neural network may take some time to train itself. Other neural network models like Kohonen Network [14] also provides good results. This approach can also be extended to recognizing handwritten paper documents. For paper documents, one can initially scan the document and convert it to binary using an adaptive threshold and then segment the lines and then find words. From the words one can find character outlines and use the back-propagation approach to identify the characters. The network can also be trained with multiple languages so that it can be used to recognize multiple languages by using different dataset for different languages.

BP and PSO Algorithms Fusion. Firstly, establish the relationship between PSO solution space and BP network structure. The solution space dimension of PSO is the dimension of particle position vector and velocity vector, which can be expressed by the combination of node numbers in each layer of BP network. Its value is the sum of the number of weights and thresholds in BP neural network. Secondly, establish the relationship between PSO fitness and the total error of BP network. The total error formula of BP is the fitness function of particles, and the fitness value of particles is the total error, which is accumulated by all training data during the forward propagation. Finally, establish the relation between PSO evolution formula and BP network weight and threshold. After each iteration, use the updated particle swarm optimal position to assign values for weights and thresholds of BP, to make the network has better convergence speed and prediction accuracy.