Computing, Information and Control ICIC International c⃝2011 ISSN 1349-4198
Volume7, Number2, February2011 pp.647–656
AN ALGORITHM OF CHAOTIC DYNAMIC ADAPTIVE LOCAL SEARCH METHOD FOR ELMAN NEURAL NETWORK
Zhiqiang Zhang1,2, Zheng Tang2, Shangce Gao2 and Gang Yang2
1Department of Research and Development
NST Inc Tokyo, Japan [email protected]
2Faculty of Engineering
Toyama University
Gofuku 3190, Toyama shi 930-8555, Japan
Received September 2009; revised January 2010
Abstract. In this paper, we present an efficient algorithm for the prediction of
sunspot-related time series, namely the Chaotic Dynamic Adaptive Local Search (CDALS) algo-rithm. This algorithm is based on exploiting partially recurrent Elman Neural Network (ENN) and it can be divided into two main steps: the first one is the basic model of the Adaptive Local Search (ALS) proposed in our previous work. After that, a hybrid local search method is proposed by introducing the chaos signals into ALS. Thus, ALS and chaos are hybridized to form a powerful CDALS algorithm, which reasonably combines the searching ability of ALS and chaotic searching behavior. Simulation results show that the CDALS algorithm can eventually reach the global optimum or its good approximation with high probability, effectively enhance the searching efficiency and quality within rea-sonable number of iterations.
Keywords: Elman neural network (ENN), Sunspot-related time series prediction, Adap-tive local search (ALS), Chaotic dynamic
1. Introduction. As we know, Elman Neural Network (ENN) consists of a two-layer
back propagation network with an additional feedback connection from the output of the hidden layer to its input. The advantage of this feedback path is that it allows the ENN to recognize and generate temporal patterns and spatial patterns. This means that after training, interrelations between the current input and internal states are processed to produce the output and to represent the relevant past information in the internal states. As a result, the ENN has been widely used in various fields, in which prediction is an important application direction [1-5].
Since the ENN usually uses the Back-Propagation (BP) based algorithms to deal with the various signals, it has been proved that it is difficult to overcome the weakness or inherent characteristics of the BP algorithm which suffers from slowness of convergence speed and easily gets stuck into the local minima. In contrast to the past learning algo-rithms employed by the ENN, we have found that all these learning algoalgo-rithms, either the error BP based algorithms or the non error BP based algorithms employed by the ENN such as genetic algorithm [6,7] and simulated annealing algorithm [8] are difficult to achieve the satisfied learning solutions that is far from the optimal solution. So these problems call for the development of alternative training algorithms for ENN.
As a novel optimization technique, chaos has gained much attention and some applica-tions during the past decades. For a given energy or cost function, by following chaotic ergodic orbits, a chaotic dynamic system can eventually reach the global optimum or
Hidden units x(k) Input units u(k) w=1.0 Context units
D
w=1.0 x(k) cOutput units y(k)
D
Figure 1. Structure of an Elman neural network model
its good approximation with high probability. On the other hand, chaos which exhibits bounded dynamic unstable, pseudo random, non-period behavior enables the search sys-tem more capable of hill-climbing and escaping from local optima [9].
To enhance the performance of the Adaptive Local Search method (ALS) proposed by us [10,11], inspired by the basic traditional local search algorithm, a hybrid local search method is proposed by incorporating chaos into ALS. Thus, ALS and chaos are hybridized to form a chaotic dynamic adaptive local search method (CDALS), which reasonably combines the searching ability of ALS and chaotic searching behavior. In this paper, a CDALS algorithm based on the logistic equation is presented to learn ENN. Simulation results and comparisons with the other algorithms show that the CDALS can effectively enhance the searching efficiency and greatly improve the searching quality within reasonable number of iterations.
The rest of the paper is organized as following: in Section 2, the structure of the ENN is described, and then the basic ALS algorithm is given. The CDALS algorithm is briefly described in Section 3. In Section 4, we apply the proposed algorithm into the prediction of sunspot-related time series to show its effectiveness. Finally we give the conclusion and expectation.
2. Adaptive Local Search Algorithm. Figure 1 shows the structure of a simple ENN.
It is easy to observe that the ENN consists of four layers: input layer, hidden layer, context layer and output layer. The activations are copied from hidden layer to context layer on
a one for one basis, generally, with fixed recurrent connection weight (w = 1.0). The
forward connection weights are trainable between hidden units and context units as other
weights. If self feedback gain aare introduced to the context unit for the ENN, it is called
modified ENN [12], and the values of the self-connection weights (a) can be changed
according to practical applications. When the value ofa is 0, the network converts to the
original ENN [13].
From Figure 1 we can see that the output of thejth context unit in the modified ENN
is given by:
where xcj(k) and xj(k) are the outputs of the jth context unit and jth hidden unit,
respectively. The value of the feedback gain a is the same for all self-connections and
it is not modified by the training algorithm. Equation (1) can be unfolded through the dynamical recurrent trace by following:
xcj(k+ 1) =xj(k) +axj(k−1) +a2xj(k−2) +a3xj(k−3) +... (2)
Equation (2) means that the output of the context unit is an integration of the output
of the hidden unit. Usually a is between 0 and 1, a value of a enables the ENN to trace
further back into the past. So the ENN can perform prediction tasks by using its good memory for the learning data.
According to our previous research work [10], we can get the learning algorithm of ALS as following rules: Vk+1 = Vk+ ∆kel if ∆M SE+<0 and ∆M SE+<∆M SE− Vk−∆kel if ∆M SE−<0 and ∆M SE−<∆M SE+ Vk otherwise (3)
Also, in order to accelerate the convergence of the proposed algorithm, we use the
parameter ∆k in different intervals of the M SE values.
∆k=βM SE 0< β < 1 (4)
where β parameter is analogous to a learning rate, and we use different β values to test
the proposed algorithm.
Equations (3) and (4) will perform the proposed algorithm iteratively and minimizes the error function along with the set of decent directions directly. Thus, the proposed
algorithm could cause the error function M SE to decrease by all parameters’ change
produced in a cycle.
In addition, for the ALS algorithm [10], in this paper, we will add some new technical supplement. As we know, the condition that the ENN architecture can employ the BP algorithm is that the weights between hidden layer and context layer are fixed at 1. But now, we delete the mechanism of the fixed weights, and use random weights in ALS algorithm since we did not adopt BP-based algorithm. Thus, the memory capacity of the ENN will be enhanced, and it should perform better than the condition of fixed weights in generalization capacity aspects for the prediction tasks.
3. Chaotic Dynamic Adaptive Local Search (CDALS) Algorithm. Although the
ALS learning algorithm described above may lead a convergence to a local minimum for the ENN, however it is not a thoroughly efficient way for the learning to reach the global minimum from the local minimum [10,14]. In order to avoid these disadvantages, we pro-pose a hybrid local search method of solving the local minimum problem by incorporating chaos into ALS and apply it into sunspots related prediction tasks.
As a kind of characteristic of nonlinear systems, chaos is a bounded unstable dynamic behavior that exhibits sensitive dependence on initial conditions and infinite unstable
periodic motions. Although it appears to be stochastic, it occurs in a deterministic
nonlinear system under deterministic conditions. In recently years, growing interests
from physics, chemistry,biology and engineering have stimulated the studies of chaos for control [15,16], synchronization [17] and optimization [18-21].
There are several chaotic systems, such as Logistic map [22], mapping drawn from chaotic neuron [23], Tent mapping [24], Lorenz System [25], Chen System [26], and so on. The first three referenced chaotic systems can show good chaotic properties. If a
0 10 20 30 40 50 60 70 80 90 100 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Figure 2. Dynamic of logistic map
designed algorithm needs the candidate points to distribute in search space as much as possible, the three chaotic systems can meet this need. It has been demonstrated that these three chaotic systems display better randomness than other systems [27,28]. We adopt the Logistic map as a studying case. Nevertheless, it does not indicate that the adopted Logistic map outperforms the other two systems. Further works can be considered regarding to this issue.
The logistic equation is defined by the following equation:
cXi(k+ 1) = 4cXi(k)(1−cXi(k)) 0< β < 1, k= 1,2... (5)
where cXi denotes the ith chaotic variable and k represents the iteration number.
Ob-viously, cXi(k) is distributed in the interval (0, 1) under the conditions that the initial
cXi(0)∈(0, 1) and that cXi(0) ̸∈ {0.25,0.5,0.75}. Figure 6 shows its chaotic dynamics,
where Xi(0) = 0.1,k = 100.
In this paper, the CDALS algorithm was developed to get a high quality solution for learning ENN networks by introducing the chaotic signal. The learning rule Equation (3) is modified by using Equation (5) as follows:
Vk+1 =Vk+⌊p1·cV(k+ 1)−p2⌋ cV(k+ 1) = 4·cV(k)·(1−cV(k)) (6)
where k denotes the iteration number, and the function ⌊x⌋ removes the fractional part
of xand returns the resulting integer value. Furthermore, ifxis negative,⌊x⌋ returns the
first negative integer less than or equal to x.
The used architectures present only one input neuron, three number of hidden units chosen by performing several selection trials and one output unit. After the network design, we initialize the weights and thresholds of the connections in [–1, 1]. Finally, the entire input sequence is presented into the network which is trained using CDALS algorithm. After training phase, we use the well trained network with steady and fixed parameters to test the prediction generalization capacity. Generally, the search procedures of the CDALS algorithm for learning ENN networks are described as Figure 3. The detailed learning algorithm proposed can be described by the following pseudo-codes:
Step 1. Pre-process the training data.
'HVLJQRIWKH (11DUFKLWHFWXUH 3UH SURFHVVLQJ WUDLQLQJ GDWD ,QLWLDOL]WLRQ 2I SDUDPHWHUV LQFOXGLQJ ZHLJKWV HWF 7UDLQJWKH WHVWLQJGDWD XVLQJWKH&'$/6 )LQLVKLQJWKH OHDUQLQJZKHQ DUULYHDWWKH HQGFRQGLWLRQ $/6DOJRULWKP &KDRVG\QDPLFV 8VLQJWKHZHOO WUDLQHGQHWZRUN ZLWKIL[HG SDUDPHWHUVWRGR SUHGLFWLRQWDVNV /HDUQLQJSURFHVV
Figure 3. General process of the proposed CDALS algorithm
Step 3. Initialize all the parameters.
Initialize the vector V (weights and thresholds) to small random values between –1 and 1 and other parameters (the maximal CPU limit time and error criterion), and set the
maximum iteration times (M ax Epoch).
Step 4. Present Inputs and Desired Outputs.
Present a training input sequence and specify the desired outputs. Step 5. Calculate Actual Outputs.
Using the employed functions in the hidden layer and the output layer to calculate the actual outputs.
Step 6. Use the CDALS learning rule Equation (6) to adapt the weights and biasing parameters.
Select a weight or a threshold randomly, and adjust it using Equation (6), If the new
solution is better than Vk or the predefined maximum iterationM ax kis reached, output
the new solution as the result of the CDALS and go to Step 7; otherwise, let k =k + 1
and repeat Step 4.
Step 7. Use the traditional ALS learning rule Equations (3) and (4) to update the parameters.
Step 8. Repeat by going to Step 4, until the window and biasing parameters stabilize. In science, chaos is used as a synonym for irregular behavior, whose long-term predic-tion is essentially unpredictable. Chaotic differential equapredic-tions exhibit not only irregular behavior but they are also unstable with respect to small perturbations of their initial condition. Consequently it is difficult to forecast the future of time series only based on chaotic differential equations, so we design the CDALS algorithm by combining the ALS and chaos with their advantages, respectively.
4. Simulations.
4.1. The design of the sunspot experiment. Solar activity is regularly monitored
by many world observatories and research centers that are able to provide the relative number of dark spots observed on the sun day after day. These data are recorded to give the so-called sunspot related time series. In the last years, there are many studies that have been made on the forecast of solar activity [29-32]. The sunspot series was the first time with series ever studied with autoregressive models, and thus has served as a benchmark in many literatures [33,34]. Consistent with these appraisals, we use the sunspot series time prediction to testify the validity of our proposed algorithm.
17000 1750 1800 1850 1900 1950 2008 20 40 60 80 100 120 140 160 180 200 Year
Number of dark spot
Yearly sunspot number from 1700 to 2008
Figure 4. The yearly sunspot number
Also, according to the common practice,the accuracy of the prediction is evaluated in terms of the Normalized Mean Squared Error (NMSE), also called by some authors the Average Relative Variance (AVR):
N M SE = 1 σ2N N ∑ i=1 (yi−y˜i)2 (7)
where yi is the actual value of the ith point of the series of length N, ˜yi is the predicted
values and σ2 denotes a variance of true time series in the prediction interval N.
In this paper, we use data provided by the Sunspot Index Data Center of the Federation of Astronomical and Geophysical Data Analysis Services [35]. It is common practice to use as the training data from 1700 to 1920 and to evaluate the performance of the model on two sets, from 1921 to 1955 (Test1) and from 1956 to 1979 (Test2). Test2 is non-stationary and is considered to be more difficult. We employ the Yearly Sunspot Number, showed in Figure 4, that contains the yearly number of dark spots from 1700 to 2008. Obviously, in order to make correct comparisons we have used the same set of values of other previous works. Also, we will compare our algorithm with numerous results found in literature.
The algorithm proposed in this paper to predict sunspot-related time series is principally based on using ENN as a prediction framework, and it is characterized by a fine data pre-processing technique in order to prepare the most suitable training sets. In this paper, for the training data pre-processing, we adopted the pre-processing technique proposed by K. Kutza [36]. The pre-processed training data of sunspot are depicted as Figure 5. 4.2. Analysis of simulation results. For the time series prediction we adopted is the Yearly Sunspot Number. We use the data from 1700 to 1920 as training data in the train-ing phase, and use two test sets to evaluate the performance of the trained network with steady learning parameters, one test set is from 1921 to 1955 and another test set is from 1980 to 1994. By testing several experiments, we have achieved the optimal performance
using an ENN with three hidden layer neurons, β as 0.5. The prediction capacity and the
comparison results with previous works are summarized in Table 1. From the simulation results of Table 1 we can see that the proposed CDALS algorithm outperformed simple AR (Auto-Regressive) models, fully recurrent neural networks DRNN (Dynamical Recurrent Neural Networks) presented in [37], the Committee Prediction method of Wan [29], the
17000 1720 1740 1760 1780 1800 1820 1840 1860 1880 1900 1920 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 Year Number of Sun
spot processed by pre−processing technique
Figure 5. Pre-processed training data from 1700 to 1920
Table 1. Comparison of generalization capacityN M SE to the same
test-ing sets for the stest-ingle step prediction
Training set Algorithms NMSE of Test1 NMSE of Test2 NMSE of Test3
AR(12) 0.427 0.966 0.238
WNet 0.086 0.350 0.219
DRNN 0.091 0.273 N/A
Training Set COMM 0.065 0.240 0.148
(1700 – 1920) RNN+BPTT 0.084 0.300 N/A
RNB+CBPTT 0.092 0.251 N/A
VGBP 0.033 0.052 0.033
Propose CDALS 0.042 0.047 0.032
recurrent networks trained with the Back Propagation Through Time algorithm (BPTT) and one of its variant, the Constructive BPTT developed in [38]. Only the Violation Guided Back Propagation technique (VGBP) [38], based on a recurrent Finite Impulse Response (FIR) network, is more accurate than our algorithm on the Test1. As we know, the prediction of Test2 and Test3 is more difficult than Test1, so we can conclude that our proposed algorithm has powerful prediction capacity and more accurate rate than those algorithms that often used as comparison object. We also gave the specific simulation curve Figure 6, in order to better understand the prediction capacity of our proposed algorithm.
For the multi-steps prediction, there are two strategies often mentioned, one is to di-rectly forecast the desired prediction horizon by training a network, another is to use the values predicted in the previous time steps as inputs for subsequent prediction. By analyzing both methods we have found that the direct approach is more suitable for our ENN. Table 2 is the mean results obtained until a prediction horizon from 2 to 6. From Table 2 we can see that our proposed CDALS algorithm has overwhelming advantage compared with existing other methods. So although the ALS is a traditional local search approach, but by introducing the chaotic dynamic signals, the CDALS exerted its robust and generalization capacity for the sunspot-related time prediction tasks.
In order to make a comparison with the most recent results published in literature about a medium-term horizon prediction of the Yearly Sunspot Number, we have computed the
Table 2. Comparison of generalization capacityM SEtestto the same
test-ing sets for all algorithms
Steps ahead Proposed CDALS RNN+BPTT RNN+CBPTT RNN+EBPTT
2 0.29 0.88 0.69 0.53 3 0.28 1.14 0.99 0.79 4 0.25 1.22 1.17 0.80 5 0.27 1.01 0.99 0.88 6 0.30 1.02 1.01 0.84 1930 1940 1950 1960 1970 1980 0 0.2 0.4 0.6 0.8 1 1.2 1.4 Year
Number of sunspots (processed)
Test1:1921−1955
Test2:1956−1979
Test3:1980−1994
1921 1994
Figure 6. Single step prediction for sunspots from 1921 to 1990
NMSEs on the cumulated test set involving Test1, Test2 and Test3. In all the multi-step predictions, the trained well ENN present the best performance, as one can see also from Figure 6, which illustrates a graph showing a comparison between the actual values and the prediction values by using the proposed algorithm.
Through above analysis of the simulation results from single prediction task to multi-steps predictions we can see that it was suitable for the ENN to train and learn the sunspots number prediction by introducing the chaos signals into ALS algorithm, and the proposed CDALS algorithm not only achieved better training results but also obtained overwhelming advantage in prediction aspects.
5. Conclusion. In this paper, we proposed a Chaotic Dynamic Adaptive Local Search
algorithm for ENN. The proposed algorithm was shown to be of high prediction capacity for the sunspot-related time series prediction than existing other algorithms. The pro-posed algorithm has been applied to the benchmark task including single prediction and multi-steps prediction of sun spots. So it could be concluded that the proposed algorithm was an effective algorithm for ENN to learning sequence time tasks. Besides that by using the CDALS algorithm to replace BP-based algorithm, calculations are easier since
no derivatives are required and local search is straightforward. However, one of the draw-back of the proposed algorithm was that the consumption of the time take longer than some algorithms , we think it is our future work direction to shorten the learning time by developing other optimization techniques.
Acknowledgment. The authors would like to thank many people who helped us in
the experiments. The authors also gratefully acknowledge the helpful comments and suggestions of the reviewers, which have improved the presentation.
REFERENCES
[1] F. Fessant, S. Bengio and D. Collobert, On the prediction of solar activity using different neural network models,Annales Geophysicae, vol.14, pp.20-26, 1995.
[2] T. Koskela, M. Lehtokangas, J. Saarinen and K. Kaski, Time series prediction with multilayer perceptron, FIR and Elman neural networks,Proc. of the 1996 World Congress on Neural Networks, 1996.
[3] P. Stagge and B. Sendhoff, An extended Elman net for modeling time series, Int. Conf. Artificial Neural Networks, LNCS, vol.1327, pp.427-432, 1997.
[4] H. He and X. Tian, An improved Elman network and its application in flatness prediction model-ing,Proc. of the 2nd International Conference on Innovative Computing, Information and Control, Kumamoto, Japan, pp.552, 2007.
[5] C. P. Lim and W. Y. Goh, The application of an ensemble of boosted Elman networks to time series prediction: A benchmark study, International Journal of Computational Intelligence, vol.3, no.2, 2007.
[6] D. T. Pham and D. Karaboga, Training Elman and Jordan networks for system identification using genetic algorithms,Artificial Intelligence in Engineering, vol.13, pp.107-117, 1999.
[7] X. Z. Gao and S. J. Ovaska, Genetic algorithm training of Elman neural network in motor fault detection,Neural Computation and Application, vol.11, pp.37-44, 2002.
[8] D. T. Pham and D. Karaboga,Intelligent Optimization Techniques: Genetic Algorithm, Tabu Search, Simulated Annealing and Neural Networks, Springer, London, 2000.
[9] B. Li and W. S. Jiang, Optimizing complex function by chaos search, Cybernetics and Systems, vol.64, pp.821-824, 1990.
[10] Z. Zhang, Z. Tang and C. Vairappan, A novel learning method for Elman neural network using local search,Neural Information Processing – Letters and Reviews, vol.11, no.8, pp.181-188, 2007. [11] Z. Zhang, G. Yang, J. Yi, Y. Zhu and Z. Tang, A new stochastic dynamic adaptive local search
algorithm for Elman neural network, International Journal of Innovative Computing Information and Control, vol.4, no.11, pp.2927-2940, 2008.
[12] D. T. Pham and X. Liu, Identification of linear and nonlinear dynamic systems using recurrent neural networks,Artificial Intelligence in Engineering, vol.8, pp.67-75, 1993.
[13] J. Elman, Finding structure in time,Trends in Cognitive Sciences, vol.14, pp.179-211, 1990. [14] C. Vairappan, S. Gao, H. Tamura and Z. Tang, An improved learning of local search for fuzzy
controller network,International Journal of Innovative Computing, Information and Control, vol.5, no.4, pp.1101-1114, 2009.
[15] Z. Tang, O. Ishizuka and K. Tanno, A learning multiple-valued logic network that can explain reasoning,T. IEE Japan, vol.119-C, no.8-9, 1999.
[16] E. Ott, C. Grebogi and J. A. Yorke, Controlling chaos,Phys. Rec. Lett., vol.64, pp.1196-1199, 1990. [17] K. Aihara, T. Takabe and M. Toyada, Chaotic neural networks,Phys. Lett. A, vol.144, pp.333-340,
1990.
[18] B. Li and W. S. Jiang, Optimizing completes function by chaos search, Cybernet System, vol.29, pp.409-419, 1998.
[19] Z. Lu, L. S. Shieh and G. R. Chen, On robust control of uncertain chaotic systems: A sliding-mode synthesis via chaos optimization,Chaos, Solutions and Fractals, vol.18, pp.819-827, 2003.
[20] L. Zhao, L. S. Shieh, G. Chen and N. P. Coleman, Simplex sliding mode control for nonlinear uncertain systems via chaos optimization,Chaos, Solutions and Fractals, vol.23, pp.747-755, 2005. [21] R. M. May, Simple mathematical models with very complicated dynamics,Nature, vol.261,
[22] L. Yang and T. Chen, Application of chaos in genetic algorithm, Communications in Theoretical Physics, vol.38, pp.168-172, 2002.
[23] P. Collet and J. P. Eckmann, Iterated Maps of the Interval as Dynamical Systems, Birkhauser, Boston, 1980.
[24] E. N. Lorenz, Deterministic nonperiodic flow, Journal of the Atmospheric Sciences, vol.20, pp.130-141, 1963.
[25] G. Chen and T. Ueta, Yet another chaotic attractor, International Journal of Bifurcations and Chaos, vol.9, pp.1465-1466, 1999.
[26] M. J. Ji and H. W. Tang, Application of chaos in simulated annealing,Chaos, Solitons and Fractals, vol.21, pp.933-941, 2004.
[27] B. Liu, L. Wang, Y. H. Jin, F. Tang and D. X. Huang, Improved particle swarm optimization combined with chaos, Chaos, Solitons and Fractals, vol.25, pp.1261-1271, 2005.
[28] E. A. Wan, Combining fossils and sunspots: Committee predictions,Proc. of the IEEE International Conference on Neural Networks, vol.4, pp.2176-2180, 1997.
[29] T. J. Cholewo and J. M. Zurada, Sequential network construction for time series prediction, Proc. of the IEEE International Joint Conference on Neural Networks, Houston, TX, pp.2034-2039, 1997. [30] M. G. Genet and A. Petrowsky, Sunspot number prediction by a conditional distribution discrimina-tion tree,Proc. of the IEEE International Joint Conference on Neural Networks, vol.1, pp.814-819, 2003.
[31] F. Liu, G. Ng and C. Quek, Rldde: A novel reinforcement learning based dimension and delay estimator for neural networks in time series prediction, Neurocomputing, vol.70, no.7-9, pp.1331-1341, 2007.
[32] H. Tamura, K. Tanno, H. Tanaka and Z. Zhang, Bi-directional function learning method for time series prediction,ICIC Express Letters, vol.2, no.4, pp.401-407, 2008.
[33] A. S. Weigend, B. A. Huberman and D. Rumelhart, Predicting the future: A connectionist approach,
Int. J. Neural System, no.1, pp.193-209, 1990.
[34] R. A. M. van der Linden and the SIDC Team,Online Catalogue of the Sunspot Index, http://sidc. oma.be/html/sunspot.html.
[35] K. Kutza, Neural Network at Your Fingertips, http://ip-atlas.com/pub/nap/nn-src, 1996.
[36] A. Aussem, Dynamical recurrent neural networks towards prediction and modeling of dynamical systems,Neurocomputing, vol.28, pp.207-232, 1999.
[37] R. Bone and M. Crucianu, An evaluation of constructive algorithms for recurrent networks on multi-step ahead prediction, Proc. of the International Conference on Neural Information Processing, pp.547-551, 2002.
[38] B. W. Wah and M. L. Qian, Time-series predictions using constrained formulations for neural-network training and cross validation,Proc. of the International Conference on Intelligent Informa-tion Processing, the 16th IFIP World Computer Congress, pp.220-226, 2000.
