Abstract: Anticipating multi-money trade rates and handling time arrangement data is regularly a critical issue in the monetary market. This paper bids the forecast of top exchanged monetary forms on the globe utilizing diverse profound learning models which incorporate top remote trade (Forex) monetary forms. This paper implements the Recurrent Neural Network models using Bidirectional RNN, Gated Recurrent Unit (GRU) and Long Short-Term Memory (LSTM) network. They foresee the transaction scale between the world's top exchanged monetary standards, for example, EUR, JPY, GBP, AUD, CAD, CHF, CNY, SEK, NZD, and INR from information by day, over 30 years to March 2019.
Index Terms: Bi-directional RNN, Deep Learning, Foreign exchange (Forex), Gated Recurrent Unit (GRU), Long Short-Term Memory (LSTM) network, Multi-currency, Machine learning, Recurrent Neural Network (RNN).
I. INTRODUCTION
As the fast development and improvement of the economy, currency markets have transformed into an essential bit of our step by step live. Properly, the stock example anticipates unbelievably been the central purposes of the open subject. Additionally, the offer exchanging association gauge is the show of attempting to take later regard an example of a securities trade. Starting late, a generous number of looks into applying AI estimations for forecast the offer exchanging framework has created. It contains various ANN tactics.
Currency is a portion instrument used in national money related to trade as a money exchange mechanical assembly. It gives a genuine impact on adjacent and worldwide financial perspectives publicize. At overall trading, the regard contained in money related benchmarks can be extraordinary, from now on it desires a base rule budgetary structures that can use all from place to place. It is an outside exchange or generally called Forex. Forex indicated as a marketplace where exchange-traded [1].
At any rate, money related time arrangement conjecture as a champion in the midst of the most testing employments of the forecast. Different expectation models have been completed using various quantifiable and handling strategies as assembled by a continuous diagram. ANNs are conceivably the most, by and large, utilized enrolling strategy
over the whole late two decades. On the whole, it is on the
Revised Manuscript Received on May 10 ,2019
Amit R. Nagpure, Department of Computer Science and Engineering, Shri Ramdeobaba College of Engineering and Management, Nagpur, India.
Dr. Avinash J. Agrawal, Department of Computer Science and Engineering, Shri Ramdeobaba College of Engineering and Management, Nagpur, India.
grounds that ANNs are data driven and are nonlinear,
requiring irrelevant assumptions about the prototypical of the problem [3].
Foreign exchange tariffs, a cash-related time arrangement significantly changes and are crazy. Deciding transformation scale instabilities are crucial to the country's economy. Various researchers nitty gritty, the three mites of representations, to be explicit stochastic representations, Artificial Neural Network portrayals, Support Vector Regression portrayals gave unreasonable guesses [2].
BP LMA Algorithm created to anticipate Multi-Currency Conversation Rates forex information orchestrates insightful structure (Ramadhani I, Jondri, Rismala R) [1]. The exploration is done to think about the exactness of stochastic, Artificial NN, Support Vector machine Regression representations in anticipating the everyday trade rates. (Nanthakumaran P., Tillakaratne C. D.) [2]. An improved ABC plots used for streamlining heaps of the Artificial Neural Network for foreseeing on various occasion arrangements. The ABC variation is a combined calculation of unique ABC with two assorted transformation techniques of DE utilized for Guessing Money Conversation Rates (Worasucheep C). [3] Discussed the distinctive standard examination methods and neural framework strategy to anticipate the cash discussion likewise, moreover, gives the effects of various topological constraints on the precision and planning the time of neural frameworks. (Gill S., Gill A., Goel N.) [5]. A half and half model got from the straight pattern display, Auto-Reverting Moving Average (MA) show, counterfeit neural system and connected hereditary calculations to the anticipation of money tariffs (Rather A. M.). [6] A half and half archetypal of GABPN with Mean Squared Error, Mean Absolute Error, (Root Mean Squared Error) developed for determining trade tariffs (Chang J, Kuan C, Lin Y.) [7].
R. Fu, Z. Zhang, L. Li, applied LSTM and GRU neural network methods for traffic flow prediction for the first time. [10]
Meita C. A., Fanny W., Suharjito, Amalia Z. recommend sentence-level SLR utilizing profound learning strategy by consolidating Bi-directional Recurrent Neural Network (BRNN), Convolutional Neural Network (CNN). [12]
This paper applies a Recurrent Neural Network using Bi-Directional Recurrent Neural Network (BRNN), Gated Recurrent Unit (GRU), and Long Short-Term Memory
Networks (LSTM)
approaches to forecast the multi-currency conversation
Amit R. Nagpure, Avinash J. Agrawal
Using Bidirectional, GRU and LSTM Neural
Network methods for Multi-Currency Exchange
rates.
II. METHODSANDDATA A.Foreign Exchange (ForEx)
Outside trade (forex) is one kind of exchange or exchange that exchanges a nation's money to other people (cash pair) including premier monetary standards showcase for 24 hrs. ceaselessly [1]. The Forex commercial center is a global scattered or over-the-counter (OTC) publicize went to the trading of money related structures. This market receives the outside transformation scale. It coordinates all pieces of acquiring, moving and exchanging budgetary norms at current or decided expenses. Concerning trading mass, it is by a prolonged shot the hugest commercial center on the planet, followed by the Acclaim publicize.
B.Multi-currency
The different sorts of multi-monetary standards are EUR, JPY, GBP, AUD, CHF, CAD, CNY, SEK, NZD, and INR. Multi-Currency Pricing (MCP) advancement is a little of a portion planning phase that styles it clear and down to earth to contribute to overall customers their own special normal cash [1]. MCP is business the executives, which empower associations to esteem retail and tasks in an assortment of extraordinary financial structures while continuing to get reimbursed and declaring in their home money. With MCP, delivery service can terrain a comparative thing for British customers in pounds sterling (GBP), French and German customers in Euros (EUR), and Japanese customers in Yen (JPY). A business gadget that empowers vendors to wander into various pieces of the worldwide business focus, MCP empowers cardholders to buy, see expenses and reimbursement in their favored money. Starting at now, this segment is available for MasterCard masterminds and Visa in a manner of speaking.
C.Deep Learning
A bit of an increasingly broad gathering of AI methodologies reliant on learning data depictions, rather than express errand computations is known as Deep learning. There are ensuing kinds of learning strategies, managed semi-directed or un-supervised then fortification. All information comprised of AI. It tries to show unique state ponders in information by various neural layers, there are diverse significant learning representations, for instance, DBNs, CNN's, RNNs. An enormous number of insightful learning portrayals have been associated with various fields and convey progressively more condition of workmanship outcomes. The basic idea of significant learning, figuring incorporates an unsupervised learning style for pre-train; train the layer in sequence, and the preparation outcome will be the commitment of subsequent layer; adjust entire layers by guiding mode. [4]
D.Recurrent Neural Network
A recurrent neural system (RNN) is a class of counterfeit neural system where associations between hubs structure a coordinated diagram along with a fleeting arrangement. This aids it to display worldly unique conduct. Not at all like feedforward neural systems, RNNs can utilize their inside state (memory) to process arrangements of information sources. This makes them relevant to assignments, for
example, unsegmented, associated penmanship recognition or discourse acknowledgment.
E.Long Short-Term Memory
An RNN done of LSTM segments is consistently called an LSTM component. A regular LSTM unit completed of a cell, a forget, an input and output gate. The cell remembers esteems over abstract time between times and the three gateways control the flood of information into, ready of the cell. It can be utilized as a compound nonlinear unit to build up a greater significant neural framework, which can reflect the effect of long-short memory has the limit of serious learning. [10]
F.Gated Recurrent Unit
The gated Recurrent unit is a gating component in repetitive neural systems, presented by Kyunghyun C. et al. The GRU resembles a long transient memory (LSTM) with overlook gate yet has fewer parameters than LSTM, as it comes up short on a yield gate. GRU's execution on specific assignments of polyphonic music, demonstrating and discourse flag displaying was observed to be like that of LSTM. GRU has been appeared to display far and away superior execution on certain littler datasets. Notwithstanding, as appeared by Gail W. and Yoav G. and Eran Y., The LSTM is "carefully more grounded" than the GRU as it can without much of a stretch, perform unbounded checking, while the GRU cannot. That's the reason the GRU neglects to learn straightforward dialects that are learnable by the LSTM. [11]
G.Bi-directional RNN
Developed in 1997 by Paliwal and Schuster, BRNN was acquainted with increment the measure of info data accessible to the system. Bi-directional Recurrent Neural Network (BRNN) associates two concealed layers of inverse bearings to a similar yield. With this type of generative profound learning, the yield layer can get data from the past (in reverse) and forthcoming (forward) states all at once.
H.Time series Prediction
It is an evaluating that uses an association of data preparation asserted as the time course of action. It might be each day, without fail, or yearly depending upon the inspiration driving data' attempting to watch. Time arrangement data used relies upon chronicled data of a distinct observation. Since the examination data in the present examination is quantitative, assessing with Time arrangement will be driven. Time arrangement expectation systems acknowledge that data or events of the prior tend to reoccur later. The urgent purpose of conjecture in Time set is what will happen, not why it occurs. [1]
I. Data
The data on the daily currency amount from the 1980s to March 2019 is used for prediction. The accessible facts are in the arrangement of overseas exchange in the method of actual volume containing Australian Dollar (AUD), Swiss Franc (CHF), Canadian Dollar (CAD), Renminbi (CNY), Euro (EUR), Pound sterling (GBP), Japanese Yen (JYP), New Zealand Dollar (NZD),
(USD). Data obtained from website https://in.investing.com with more than 12000 total data from over 30 years for respective currency and data torn apart into training and testing, into the percentage of 80-20%.
III. SYSTEMDESIGN A. Overview
In this paper, application of the type of multi-currency conversation tariffs and implementing recurrent neural network algorithms by means of deep learning using the currency pairs, such as EUR, JPY, GBP, AUD, CAD, CHF, CNY, SEK, NZD, and INR with respect to the USD as the base currency.
B. Pre-processing
Pre-processing completed by handling absent data with the interpolation technique and scaled with the minmaxscaling scheme. Features changed by mounting every single part to a certain range. Scales and construes to each element independently assumed pitch on the training data.
The conversion is given by:
(1)
(2)
Where, min, max = feature range.
This translation often used as another to zero resources, unit adjustment scaling.
C. The long short term memory
There are a couple of structures of LSTM units. Common designing made of a Cell, the memory part of the LSTM unit and three "controllers," all the more regularly called entryways, on the flood of information inside the LSTM unit: a forget, an input and output gate. A couple of assortments of the LSTM unit don't have no less than one of these entryways or maybe have distinctive gateways. Instinctually, the cell is in charge of checking the conditions between the parts in the info gathering. The information entryway controls how much another regards streams into the cell, the overlooked door controls how much regard remains in the cell, and the yield entryway controls how much the motivating force in the cell is used to process the yield activation of the LSTM unit. The commencement limit of the LSTM doors is every now and again the key limit.
[image:3.595.318.557.52.217.2]There is relationship into and out of the LSTM doors, a couple of which are discontinuous. Heaps of these connections, which ought to hang off in the midst of getting ready, choose how the doors work.
Fig. 1: The Long Short-Term Memory (LSTM) cell [11]
: Layers
: Pointwise OP
: Copy
The equations for a forward pass of an LSTM are as follows:
(3)
(4)
(5)
(6)
(7)
σg: a function of type sigmoid
σh: a function of type hyperbolic tangent
: denotes the Hadamard productW, U, b: weight matrices and bias vector parameters. xt: input vector
ft: forget gate's vector it: input gate's vector ot: output gate's vector ht: hidden state vector ct: cell state vector
D. Gated Recurrent Unit
The GRU RNN lessens the gating signs for two from the LSTM RNN show. The two doors are called an update entryway zt and a reset door rt. The GRU RNN display is exhibited in the structure:
(8)
(9)
With the two doors
(10)
(11) [image:4.595.106.233.250.319.2]
One sees that the GRU RNN i.e Equations (8) and (9) is comparative to the LSTM RNN i.e. Equations (3) and (7). This spares one gating signal and the related parameters. Specifically, the complete number of parameters in the GRU RNN equals 3 × (n2+nm+n). In different examinations, it has been noticed that GRU RNN is practically identical to, or even beats, the LSTM much of the time. Also, there are additional diminished gated RNN, the Minimal Gated Unit (MGU) RNN, where just a single door condition is utilized and it is accounted for, that this (MGU) RNN execution is equivalent to the GRU RNN, and by deduction, to the LSTM Recurrent NN.
Fig. 2. Gated Recurrent Unit [11]
E. Bi-directional RNN
Bi-directional RNN is a joined RNN that keeps running, advancing through time as of the begin as far as possible in a succession and afterward runs in reverse over the period from the conclusion to the start. The bidirectional-RNN engineering outlined in Fig. 3. Bidirectional-RNN design is utilized to learn and delineate information arrangement x.
Fig. 3. Bi-directional RNN Architecture [12]
A state h(t) is as of sub-RNN, pushes ahead through the period, a state g(t) is as of sub-RNN that goes in reverse Unit yield o(t) is utilized to check the portrayal reliance in the past and the following state.
F. Performance Measurement 1) Loss function:
A loss function is one of the two limitations required to collect a model.
a) Mean Absolute error
The Mean Absolute Error (MAE) is a proportion of contrast between two persistent factors.
( 1 2 )
b) Mean Squared error
The Mean Squared Error (MSE) is the normal of squared blunder that utilized the misfortune work for least squares relapse.
(13)
Likewise, pass the identifier of a present misfortune capacity or grant a TensorFlow delegate work that benefits a scalar for each measurement point and receipts the going with two parameters:
y_true: True names. Tensor.
y_pred: Predictions. Tensor of the indistinguishable system from y_true.
The authentic improved target is the mean of the yield, group in general information focuses.
2) Optimizer: Adam
An algorithm projected by Kingma and Lei Ba for the first-order gradient built optimization of stochastic neutral function known as Adam optimizer is built on the adaptive estimation of trivial-order moments, in Adam.
3) Dropout
Deep learning system representations in overall have an enormous number of constraints. Overfitting is dependably an issue in utilizing this sort of system Dropout is a procedure connected to look out of this matter Dropout means to cut the association drop component among neuron part and limit the state of the system during training The detachment of neuron stops the system to be excessively sharp on the preparation information gave Besides dropout gives an approach to consolidate neural system designs all the while and productively.
4) The coefficient of determination (R2)
The coefficient of determination, R2, is used to separate how a refinement in a second factor can elucidate differentiates in a solitary variable. For example, when an individual gets pregnant has a quick association with when they imagine a posterity. Significantly more expressly, R-squared gives the rate assortment in y cleared up by x-factors. The range is 0 to 1, i.e., The x-components can determine 0% to 100% of the cluster in y. The coefficient of assurance, R2, resembles the connection coefficient, R.
The association coefficient formula will uncover how strong of an immediate relationship there is among two variables. R2 is the square of the convection coefficient, R.
R2 is given by as pursues:
(14)
yi: n values of data fi: Predicted value
y
: observed data’s mean [image:4.595.103.244.446.576.2]The model constructed on the framework of python, and it uses keras deep learning framework. After training the arrangement repeatedly, the following constraints established for the top outcomes. A BRNN uses 3 hidden layers with 512,256,1 neurons in each layer. A GRU uses 3 hidden layers with 256,512,1 neurons in each layer. An LSTM uses 7 neurons in a hidden layer. Adam optimizer, ReLU is used as activation, lecun uniform for kernel initializer, with 100 epochs and a batch dimension of 32 for BRNN, GRU, and LSTM. As the data used for this consists of over 30 years of day by day data, separated into training and testing in 80%-20% percentage from the 1980s till March 2019, the data since January 2011-March 2019 is for testing, and the following are the outcomes based on that dataset.
Mean
Std
25%
50%
75%
EUR
0.876
0
0.155
6
0.768
8
0.839
9
0.9305
JPY
112.0
6
16.19
102.8
1
111.8
4
121.86
GBP
0.623
2
0.080
9
0.573
6
0.624
2
0.6665
AUD
1.306
6
0.242
5
1.138
3
1.309
7
1.4196
CAD
1.264
9
0.156
0
1.158
5
1.271
7
1.3707
CHF
1.257
5
0.248
4
1.001
7
1.256
1
1.4665
CNY
6.163
5
2.126
9
4.733
9
6.589
9
8.2768
SEK
7.494
4
1.148
6
6.599
5
7.375
8
8.2143
NZD
1.590
0
0.310
8
1.379
7
1.527
9
1.7889
MX
N
10.33
2
4.568
8
7.820
6
10.71
8
12.955
4
INR
36.40
5
18.28
5
16.77
41.72
5
47.435
Table 1. Dataset Statistics
Currency Training Error Testing Error R
2
Score
MSE MAE MSE MAE
EUR 6.08e0
5 0.006 4 4.09e0 5 0.005 5 0.994
JPY 6.79e0
5 0.005 4 1.81e0 5 0.007 7 0.995
GBP 5.98e0
5 0.006 1 7.73e0 5 0.007 3 0.995
AUD 4.38e0
5 0.004 1 2.57e0 5 0.003 6 0.999
CAD 5.85e0
5 0.005 7 5.31e0 5 0.005 2 0.999
CHF 3.94e0
5 0.016 5 4.53e0 5 0.004 2 0.982
CNY 5.95e0
5 0.001 1 2.16e0 5 0.001 1 0.998
SEK 5.77e0
5 0.005 0 4.66e0 5 0.004 6 0.998
NZD 6.98e0
5 0.005 8 3.38e0 5 0.004 5 0.994
INR 8.95e0
5 0.001 8 3.86e0 5 0.005 4 0.995
Table 2. Evaluation Result of Bi-directional Recurrent Neural Network
Currency Training Error Testing Error R
2
Score
MSE MAE MSE MAE
EUR 9.41e0
5 0.008 4 9.50e0 5 0.008 9 0.986
JPY 8.11e0
5 0.006 5 1.99e0 5 0.009 3 0.994
GBP 6.70e0
5 0.006 9 6.86e0 5 0.006 9 0.995
AUD 5.91e0
5 0.005 8 4.54e0 5 0.005 6 0.997
CAD 8.36e0
5 0.007 7 9.04e0 5 0.007 9 0.998
CHF 1.14e0
5 0.008 1 4.03e0 5 0.003 8 0.984
CNY 1.65e0
5 0.002 8 2.54e0 5 0.001 2 0.998
SEK 6.11e0
5 0.005 3 4.84e0 5 0.004 9 0.998
NZD 6.15e0
5 0.005 2 4.10e0 5 0.005 2 0.993
INR 3.86e0
[image:5.595.47.549.160.805.2]5 0.005 4 1.06e0 5 0.009 0 0.996
Table 3. Evaluation Result of Gated Recurrent Unit
Currency Training Error Testing Error R
2
Score
MSE MAE MSE MAE
EUR 2.56e0
5 0.003 3 1.58e0 5 0.002 6 0.998
JPY 6.76e0
5 0.005 4 3.79e0 5 0.010 6 0.989
GBP 3.26e0
5 0.003 8 3.21e0 5 0.003 6 0.998
AUD 4.30e0
5 0.003 9 2.42e0 5 0.003 4 0.999
CAD 3.88e0
5 0.004 0 4.67e0 5 0.004 7 0.999
CHF 8.12e0
5 0.006 2 1.21e0 5 0.008 6 0.998
CNY 5.67e0
5 0.000 9 3.38e0 5 0.001 5 0.997
SEK 5.96e0
5 0.005 1 4.73e0 5 0.004 7 0.998
NZD 4.84e0
5 0.003 9 2.41e0 5 0.003 4 0.996
INR 5.56e0
5 0.001 2 1.19e0 5 0.008 0 0.996
Fig. 4. Result of EUR rates Prediction
[image:6.595.78.270.54.211.2]Fig. 5. Result of JPY rates Prediction
Fig. 6. Result of GBP rates Prediction
Fig. 7. Result of AUD rates Prediction
[image:6.595.338.525.216.372.2]Fig. 8. Result of CAD rates Prediction
Fig. 9. Result of CHF rates Prediction
Fig. 10. Result of CNY rates Prediction
[image:6.595.340.526.406.557.2] [image:6.595.81.269.582.730.2]Fig. 12. Result of NZD rates Prediction
Fig. 13. Result of INR rates Prediction
V. CONCLUSION
This paper predicts the exchange rates among world’s trafficked currencies EUR, JPY, GBP, AUD, CAD, CHF, CNY, SEK, NZD, and INR from data day by day, above 30 years till March 2019. The above effort validates the relevance of the RNN uses Bi-directional RNN, GRU and LSTM to the problem of multi-currency conversation rate forecast. The conclusions are deeply auspicious; results displayed from the values of Loss functions (MSE and MAE) along with the score of Coefficient of Determination (R2) that the normal accurateness of all the predicting model, BRNN, GRU, and LSTM as a maximum of 99 %.
REFERENCES
1. Ramadhani Imaniar, Jondri, Rismala Rita, “Prediction of Multi-Currency Exchange Rates Using Correlation Analysis and Backpropagation,” International Conference on ICT for Smart Society Surabaya, 20-21, 978-1-5090-1620-4, July 2016.
2. Nanthakumaran P., Tilakaratne C. D., “A Comparison of Accuracy of Forecasting Models: A Study on Selected Foreign Exchange Rates,” Evolutionary Computation (CEC), 2015 IEEE Congress, September 2015.
3. Worasucheep Chukiat, “Forecasting Currency Exchange Rates with an Artificial Bee Colony (ABC) Optimized Neural Network,” International Conference on Advances in ICT for Emerging Regions (ICTer): 324 – 331, 2017.
4. Gao Tingwei, Li Xiu, Chai Yueting, Tang Youhua “Deep Learning with Stock Indicators and Two Dimensional Principal Component Analysis (PCA) for Closing Price Prediction System,” 4673-9904-3/16/$31.00, 2016 IEEE.
5. Gill S.S., Gill Amanjot Kaur, Goel Naveen, “Indian Currency Exchange Rate Forecasting Using Neural Networks,” 4244-6932-1/10, 2010 IEEE.
6. Rather Akhter Mohiuddin, “Computational intelligence based hybrid approach for forecasting currency exchange rate,” 4799-8349-0/15, 2015 IEEE.
7. Chang Jui-Fang, Kuan Chi-Ming, Lin Yu-Wen, “Forecasting Exchange Rates by Genetic Algorithms Based Back Propagation Network Model,” 978-0-7695-3762-7/09, 2009 IEEE.
8. BIS, “Triennial Central Bank Survey - Foreign Exchange Turnover in April 2013: Preliminary Global Result”, Bank Int. Settlements Rev., No. April, p. 24, 2013.
9. Wang Tien-Chin, Kuo Su-Hui, Chen Hui-Chen, “Forecasting the Exchange Rate between ASEAN Currencies and USD,” 4577-0739-1/11, 2011 IEEE.
10. Rui Fu, Zuo Zhang, Li Li, “Using LSTM and GRU neural network methods for traffic flow prediction”, 978-1-5090-4423-8/16/$31.00, 2016, IEEE.
11. Xiaoyun Qu, Xiaoning Kang, Chao Zhang, Shuai Jiang, Xiuda Ma, “Short-Term Prediction of Wind Power Based on Deep Long Short-Term Memory,” 5090-5417-6/16/ $ 31.00, 2016, IEEE. 12. Meita Chandra Ariesta, Fanny Wiryana, Suharjito, Amalia Zahra
“Sentence Level Indonesian Sign Language Recognition Using 3D Convolutional Neural Network and Bidirectional Recurrent Neural Network”, 978-1-5386-9422-0/18/$31.00 2018 IEEE.
13. Amit R. Nagpure, “Prediction of Multi-Currency Exchange Rates Using Deep Learning”, International Journal of Innovative Technology and Exploring Engineering (IJITEE), Vol.8 (6), ISSN:2278-3075, 2019, 316-322.
14. Nishchal Puri, Dr. Avinash Agrawal, Prakash Prasad, “A Survey on Machine Learning Approach for Stock Market Prediction”, Helix the Scientific Explorer, Vol.8 (5), ISSN 2319-5592, 2018, 3705-3709. 15. N. S. Puri, Dr. A. J. Agrawal,” A Technique for Stock Prediction based
on Textual Analysis of Financial Articles”, International Journal of Advanced Engineering, Management and Science (IJAEMS), Special Issue 3, ISSN 2454-1311, 2017.
AUTHORSPROFILE
Amit R. Nagpure did Bachelor of Engineering in Computer Engineering from Rashtrasant Tukadoji Maharaj, Nagpur University. Currently pursuing Master of Technology in Computer Science and Engineering from Shri Ramdeobaba College of Engineering and Management, Nagpur, India, IEEE member under Nagpur Subsection under R10 IEEE region, participated in IEEE student website contest, completed basic certifications in Data Science from IBM’s cognitive.ai online portal.
[image:7.595.81.267.59.232.2]![Fig. 1: The Long Short-Term Memory (LSTM) cell [11]](https://thumb-us.123doks.com/thumbv2/123dok_us/8204304.261735/3.595.318.557.52.217/fig-long-short-term-memory-lstm-cell.webp)
![Fig. 2. Gated Recurrent Unit [11]](https://thumb-us.123doks.com/thumbv2/123dok_us/8204304.261735/4.595.103.244.446.576/fig-gated-recurrent-unit.webp)
