Development of Improved Artificial Neural
Network Model for Stock Market
Prediction
PRATAP KISHORE PADHIARY1
M.Tech (IT) Scholar, S’O’A University, Bhubaneswar, India.
AMBIKA PRASAD MISHRA2
Asst. Professor, Dept. of CSE, S’O’A University, Bhubaneswar, India.
Abstract:
In recent year’s prediction of stock market returns is a hottest field of research in finance. Artificial Neural Network (ANN) is a technique that is heavily researched and widely used in applications for engineering and scientific fields for various purposes ranging from control systems to artificial intelligence. This paper surveys key issues in financial forecasting and propose an ANN methodology which could be better for long term (one month, two month) as well as short term (one day) prediction of stock price of any leading stock market indices. Survey of existing literature reveals that adaptive learning rate will give more accurate result than fixed learning rate parameter for ANN models. Many researchers noted that slight parameter changed causes major variations in the behavior of the network. So there is no theory which could be guideline for finding best network topology. The proposed trigonometric functional link artificial neural network (FLANN) model employs standard least mean square (LMS) algorithm with search-then-converge scheduling which could effectively calculate learning rate parameter that changes with time and may require less experiments to train the model. The objective of this paper is to introduce a functional link single layer artificial neural network (FLANN) for long term as well as short term stock market prediction.
Keywords: Neural Network, FLANN, stock prediction, Learning rate, LMS, search-then-converge
1. Introduction
Stock price prediction is an important field of research in finance because if the market is successfully predicted then the investors may get maximum returns. The stock market or equity market is a public market where a large amount of capital are invested and traded in everyday all over the world. Many researchers claim that the market is dynamic, non-linear, complicated and chaotic in nature. So it is difficult to deal with normal analytical methods like time series analysis. These chaos systems are sensitive to initial conditions. So the neural networks are effective to deal with such a non-linear system. However financial time-series are difficult to forecast because these are noisiest and non-stationary signals [Oh and Kim, (2002)]. Some common financial time-series are currency exchange rates, interest rates, stock prices etc. A number of researchers have given their view on Efficient Market Hypothesis (EMH) [Lowe and Webb, (1991)]. EMH states that the market is efficient so it cannot be predicted because when new information arises, the market corrects itself and absorbs it [Malkiel, (1999)]. There is no such information to predict the market in such a way that the investors earn greater profits from stock market. In the recent years, many researchers claimed that the EMH must be false. From the last years many researches are on this field, still it remains a big task whether the market can be correctly predicted or not. To predict the stock market accurately, various prediction algorithms and models have been proposed by many researchers in both academics and industry.
2. Literature Survey
The stock market is volatile because of many unseen factors that can influence the share price. There are two types of factors: Qualitative and Quantitative. Qualitative factors include political events, international ‘events’ [Ng and Fu, (2003)], firms’ policies etc. and Quantitative factors such as open rate, close rate, high rate, and low rate for individual equities. A stock market prediction system was designed by [Kohara et al., (1997)] using the prior knowledge and event-knowledge. They incorporated prior-knowledge in stock prediction such as newspaper information on domestic and foreign events. Event-knowledge is extracted from the news paper headlines in accordance with certain prior-knowledge. Prior knowledge is the information that stems from previous experience. Thus, based on the prior-knowledge, decisions can be made whether a particular event can positively influence the stock market tendencies or not. [Hong and Han, (2004)] introduced an automated system (KBN Miner) that extracts event-knowledge from the Internet for the prediction of interest rates. The KBN Miner which is based on a prior-knowledge collect the event information from the Internet automatically and helps in decision making whether a particular event can positively affect the stock market tendencies or not and then to apply the information to a neural network model for interest rate prediction. Web mining technique that they applied for predicting interest rates can also be applied for stock market prediction. In finance, technical analysis is a method of security analysis which forecasts general price direction by analyzing market activity such as past price and volume. In this process technical analyst use technical indicators and chart patterns from historical data.Yao,Tan and Poh [Yao et al.,(1999)] suggest that technical analysis is not appropriate for market prediction as it is highly volatile and the use of technical indicators would be widely adopted by traders. Most common indicators are such as moving average, relative strength index, stochastic oscillator etc. For such reasons researchers have stressed on developing models for accurate prediction based on various statistical and soft computing techniques. Auto-regressive integrated moving average (ARIMA) based model [Schumann and Lohrbach, (1993)] is one of the most important and widely used statistical technique employed in this regard. The ARIMA model is obtained by differentiating an assumed non-stationary process to obtain a locally wide sense stationary (wss) and locally ergodic process. It applies the Box–Jenkins methodology [Box and Jenkins, (1970)] in the model building process. The ARIMA model is an efficient approach but with short comings like correlation analysis.
Recent advances in soft computing led to a new era in the field of financial forecasting. In the most recent times the soft computing tools based on such as multilayer artificial neural networks (ANN) [Kingdon, (1997); Refenes, (1995); Ziurilli, (1997)], Fuzzy logic (FL) [Ju et al., (1997)], Genetic Algorithm (GA) [Bhattacharya and Meheta,(1998)], genetic programming (GP) [Neely,(1997)] and hybrid tools [Hassan,(2009); Hassan et al.,(2007); Kim,(2006);Versace et al.,(2004)] have been applied to financial forecasting. Learning algorithms such as support vector machine has been employed [Huang et al., (2005)] for forecasting stock market movement direction. NNs with GA have been used to predict the Singapore Stock Exchange Index and achieved accuracy rate of 81% [Phua et al., (2000)].Kim and Han [Kim and Han, (2000)] also combined NNs with GA and predicted Korea Composite Stock Price Index 200. He achieved 82% of accuracy in predicting both weekly rising and declining stock market tendencies. In some cases, several economic indicators such as interest rate, price of crude oil, and New York Dow Jones average of the closing price are selected and fed them together with event-knowledge into NNs. Their experimental results showed incorporation of event knowledge improved the prediction ability of NNs by reducing the error rate on the 5% level of significance. A study has been made to investigate appropriate selection and effects of various network parameters in the design of back propagation neural network model [Tan Clarence and Wittig, (1993)].The parameters they studied are learning rate, momentum, input noise, number of hidden layers and activation function. Tools based on ANN have increasingly gained popularity due to their inherent capabilities to approximate any nonlinear function to a high degree of accuracy. Neural networks are less sensitive to error term assumptions and they can tolerate noise and chaotic components [Masters, (1993)].
[Garliauskas, (1999)] concluded that in predicting financial time series, NNs have better performance than classical statistical methods.
The present paper of our interest is to develop a low complexity and accurate prediction model which is better suited for long term prediction. The objective is to introduce a functional link single layer artificial neural network (FLANN) for developing efficient stock market prediction model using LMS learning rule.
3. Neural Network
A neural network is a massively parallel distributed processor made up of simple processing unit which has a natural propensity for storing experiential knowledge and making it available for use [Haykin]. Each connection between units has an associated real-valued weight which simulates the efficacy of biological synapses in the brain. Given a set of labeled input-output pairs, these models accomplish the learning process by adaptively adjusting their parameters (weights) in such a way as to perform a given task. Neural networks have remarkable ability to derive meaning from complicated or imprecise data. They are used to extract patterns and detect trends that are too complex to be noticed by either humans or other computer techniques. The ability of neural networks to discover nonlinear relationships [Phillip and Nostrand, (1989)] in input data makes them ideal for modeling nonlinear dynamic systems such as the stock market. They have remarkable ability to derive meaning from complicated or imprecise data can be used to extract patterns and detect trends that are too complex to be noticed by either humans or other computer techniques. A neural network method can enhance an investor's forecasting ability [Youngohc and George, (1991)]. Neural networks are also gaining popularity in forecasting market variables [Hamid, (2004)].Neural network have good generalization capability and usually robust against noisy or missing data, all of which are highly desirable properties for time series prediction.
The limitation of this approach is that such models are black box in nature that means models do not capture the cause of the movements of stock prices in the market. Another serious problem with NNs is the overfitting problem [Haykin]. It occurs when the network has too many free parameters which allow the network to fit well the training data but typically lead to poor generalization. Overfitting occurs because of two main reason, first is if the network have too many nodes and the second is if the network trained more than necessary. Moreover, NNs have some limitations in learning the patterns when input data have high dimensionality. Dash and Liu [Dash and Liu,(1997)] put the emphasis on the feature selection and suggested that reducing the number of input variables sometimes lead to improved model performance for a given data set. The reduction and transformation of the irrelevant or redundant features may shorten the running time and yield more generalized results.
4. FLANN Model
functional approximation, and digital communications channel equalization. The proposed model consists of three basic processes.
4.1. The functional expansion (FE) process
Here input elements are nonlinearly expanded to generate more number of inputs. The Process should be nonlinear and should involve simple computations. This preliminary introduction of nonlinearity reduces the number of layers and computational complexity.
4.2. The estimation process
It include two steps (i) computing the output of the adaptive model in response to nonlinear input elements (ii) comparing the estimated financial output of the model with the corresponding desired or target result to generate error signal.
4.3. Adaptive process
Here the model updates the connecting weights by means of some adaptive learning rule. The combination of these three processes together constitutes the proposed financial model as shown in Fig. 1.
Fig. 1. Adaptive FLANN Model
5. FLANN Model Training by Gradient Descent Method
Steepest descent method is a statistical optimization technique for minimizing a function of several variables. Successive adjustments of the weights are in the direction of steepest descent, that is, in the opposite direction of gradient vector (w).
For convenience of presentation we write g= (w)
By differentiating cost function with respect to weight vector.
We get g(k) = –e(k)x(k) (1) Where, g(k) = gradient vector at kth nexperiment
e(k) = error at kth experiment x(k) = input vector at kth experiment The steepest descent algorithm is described by
w (k+1) = w (k) – µg (k) (2) Updating the weights using steepest descent method as
w (k+1) = w(k) + µe(k)x(k) (3) Where, µ=learning rate parameter between 0 and 1.
w (k+1) = updated value of weight vector at k+1 experiment. w (k) = old value of weight vector at kth experiment.
e (k) = error at kth experiment. x (k) = input vector at kth experiment.
6. Proposed Approach towards Improved FLANN Model
implementing adaptive learning using search-then-converge rule Darken and Moody [Darken and Moody, (1990b)]. The adaptive learning rate provides better result in terms of accuracy, speed of convergence and stability. Here learning rate parameter change with time. Learning rate parameter is large at the beginning of training and gradually decreases as the network converge. The rule consists of two phases. The first phase is searching phase where learning rate parameter is large and almost constant and second phase is the converging phase where learning rate parameter decrease exponentially to zero. Then the parameters used in the FLANN model can be computed and updated using LMS algorithm [Fredric and Ivica] which comprises of following steps:
1. Set k=1, initialize the synaptic weight vector w (k=1)& select values for µ0 and τ.
where
2. Compute
Where µ0and τ are user selected constants and 100≤ τ ≥ 500
3. Compute the error
4. Update the synaptic weights as following
wi(k+1) = wi(k) + µ(k) e(k) xi(k), for i=1,2,……n.
5. If convergence is achieved, stop; else set k←k+1, then go to step 2.
7. Comparison of Learning Rate Schedules
Fig 2 shows a comparison of the stochastic approximation and search-then-converge schedule for the LMS Learning rate parameter. In standard LMS algorithm the learning rate parameter is constant for all simulation.So it requires more number of training to converge and in stochastic approximation schedule the learning rate decreases rapidly. In search- then-converge, the learning parameter is large at the beginning of the training and decreases gradually as the network converges. So we go for Search-then-Converge schedule.
Fig.
2. Learning rate schedule comparison
8. Conclusion & Future Work
References
[1] Bhattacharya, S., & Meheta, K. (1998). Staged learning of trading rules using genetic algorithm. In Proceedings of 3rd
INFORMS conference on info. sys. and tech (pp. 62–66) Montreal, April.
[2] Box, G.E.P., Jenkins, G.M. 1970. Time Series Analysis, Forecasting and Control, Holden-Day, San Francisco,CA,
[3] Darken C. and Moody, J. (1990b) Note on learning rate schedules for stochastic optimization. Advances in Neural Information Processing Systems 3, Morgan Kauffman, San Mateo, California. 832-838
[4] Dash, M. and Liu, H. 1997, ‘Feature selection for classifications’, Intelligent Data Analysis: An International Journal, vol. 1, pp. 131-156.
[5] Fredric M. Ham, Ivica Kostanic, Principle of Neurocomputing for Science and Engineering, 1st Edition, Tata-McGrawhill Edition [6] Garliauskas, A. 1999, ‘Neural Network Chaos and Compuational Algorithm of Forecast in Finance’, Proceedings of the IEEE SMC
Conference on Systems, Man, and Cybernetics 2, pp. 638-643, 12-15 October.
[7] Hong, T. and Han, I. 2004, ‘Integrated approach of cognitive maps and neural networks using qualitative information on the World Wide Web: KBN Miner’, Expert Systems, vol. 21 no.5, pp. 243-252.
[8] Haykin, Simon, Neural Networks, A Comprehensive Foundation, 2nd Edition, Prentice Hall International.
[9] Hassan, Md., (2009). A combination of hidden Markov model and fuzzy model for stock market forecasting. Neurocomputing Elsevier, 72, 3439–3446
[10] Hassan, Md., Nath, B., & Kirley, M. (2007). A fusion model of HMM, ANN and GA for stock market forecasting. Expert System with Applications, Elsevier, 33(1), 171–180, July.
[11] Huang, W., Nakamori, Y., & Wang, S. (2005). Forecasting stock market movement direction with support vector machine. Computer and Operation Research, Elsevier, 32, 2513–2522.
[12] Ju, Y., Kim, C., & Shim, J. C. (1997). Genetic–based fuzzy models; interest rates Forecasting problem. Computer and Industrial Engineering, 33, 561–564.
[13] Kohara, K. Ishikawa, T. Fukuhara, Y. and Nakamura, Y. 1997, ‘Stock Price Prediction Using Prior Knowledge and Neural Networks’, Intelligent System In Accounting, Finance and Management, vol. 6, pp. 11-22 vol.16.
[14] Kingdon, J. (1997). Intelligent systems and financial forecasting. Berlin: Springer-Verlag.
[15] Kim, K. (2006). Artificial neural networks with evolutionary instance selection for financial forecasting. Expert System with Applications, Elsevier, 30, 519–526.
[16] Kim, K. and Han, I. 2000, ‘Genetic algorithms approach to feature discretization in artificial neural networks for the prediction of stock price index’, Expert System Appliance, vol. 19.
[17] Lowe, D and Webb, A. R. 1991, ‘Time series prediction by adaptive networks: A dynamical systems perspective. IEEE Computer Society Press.
[18] Lee, R. S. T. (2004). IJADE stock advisor: An intelligent agent based stock prediction system using hybrid RBF recurrent network. IEEE Transactions on Systems, Man and Cybernetics, Part A, 34(3), 421–428.
[19] Lippman, R. P. (1987). An introduction to computing with neural nets.IEEE ASSP Magazine, 4, 4–22. [20] Masters, T. (1993). Practical neural network recipes in C++. New York: Academic Press.
[21] Malkiel B.G, “A Random Walk Down Wall Street”, W. W. Norton &Company, New York, London, 1999.
[22] Ng, A. and Fu, A.W. 2003, ‘Mining Frequent Episodes for Relating Financial Events and Stock Trends’, Lecture Notes in Computer Science, vol. 2637, pp. 27-39.
[23] Neely, C. (1997). Is technical analysis in the foreign exchange market profitable: A genetic programming approach. Journal of Financial and Quantitative Analysis, 32(4).
[24] Oh, K. J., & Kim, K-J (2002). Analyzing stock market tick data using piecewise non linear model. Expert System with Applications, 22, 249–255.
[25] Ornes C. & Sklansky J. (1997). A neural network that explains as well as predicts financial market behavior. In Proceedings of computational intelligence for financial engineering, the IEEE/IAFE 1997 (pp. 43–49).
[26] Masters, T. (1993). Practical neural network recipes in C++. New York: Academic Press.
[27] Phillip D. Wasserman, Van Nostrand "Neural Computing: Theory and Practice", Van Nostrand Reinhold, New York, 1989 [28] Pao, Y. H. (1989). Adaptive pattern recognition & neural networks. Reading, MA: Addison-Wesley.
[29] Patra, J. C., Pal, R. N., Chatterji, B. N., & Panda, G. (1999). Identification of nonlinear dynamic systems using functional link artificial neural networks. IEEE Transactions on systems, man and cybernetics-Part B: Cybernetics, 29(2), 254–262.
[30] Phua, P. K. H. Ming, D. and Lin, W. 2000, ‘Neural Network with Genetic Algorithms for Stocks Prediction’, Fifth Conference of the Association of Asian-Pacific Operations Research Societies, 5th - 7th July, Singapore.
[31] Refenes, A. (1995). Neural network in financial engineering. In Proceedings of fourth international conference neural networks in the capital market (NNCM-95) (pp.Singapore). World Scientific: WorldScientific.
[32] Saad, E. W., Prokhorov, D. V., & Wunsch, D. C. (1998). Comparative study of stock trend prediction using time delay, recurrent and probabilistic neural networks. IEEE Transactions of Neural Network, 9(6), 1456–1470.
[33] Shaikh A. Hamid. Primer on using neural networks for forecasting market variables. In proceedings of the a conference at school of business, southern new hampshire university, 2004.
[34] Tan Clarence, N. W., & Wittig Gerhard, E. (1993). A Study of the parameters of a backpropagation stock price prediction model. In Proceedings of first New Zealand international two-stream conference on artificial neural networks and expert systems (pp. 288–291). [35] Tan H., Prokhorov D. V. & Wunsch D. C., II (1995). Conservative thirty calendar day stock prediction using a probabilistic neural
network. In Proceedings of computational intelligence for financial engineering, the IEEE/IAFE 1995 (pp.113–117).
[36] Versace, M. et al. (2004). Predicting the exchange traded fund DIA with a combination of genetic algorithms and neural networks. Expert Systems with Applications, Elsevier, 27, 417–425.
[37] Wang, Y. (2003). Mining stock prices using fuzzy rough set system. Expert System with Applications, 24, 13–23.
[38] Yao, J., Tan C.J., Poh H.L., Neural networks for technical analysis: a study on KLCI, International Journal of Theoretical and Applied Finance 2 (2) (1999).
[39] Yamashita T., Hirasawa K. & Hu J. (2005). Application of multi-branch neural networks to stock market prediction. In Proceedings of IEEE international joint conference on neural networks (IJCNN ‘05) (Vol. 4, pp. 2544–2548).
