Table 4.1: MIT-BIH Arrythmia database recording names used for training
(DS1) and testing (DS2) datasets. Records in gray do not contain precordial
lead V1 and are extracted from the datasets.
Dataset MIT-BIH Arrhythmia record names
DS1 101, 106, 108, 109, 112,114, 115, 116, 118, 119, 122,
124, 201, 203, 205, 207, 208, 209, 215, 220, 223, 230. DS2 100,103, 105, 111, 113,117, 121,123, 200, 202, 210, 212, 213, 214, 219, 221, 222, 228, 231, 232, 233, 234.
real-world classification problem described in Chapter 3. We apply a similar multivariate analysis to study the classification ofECGsignals when a single or two channels are taken into account to classify healthy and pathological subjects with a delay-based reservoir.
4.2
Multivariate arrhythmic heartbeat classification with
logistic regression
This task consists on the classification of heartbeats from the MIT-BIH Arrhyth- mia database. The univariate classification problem is described in detail in Chapter3.
In the following, we choose the MIT-BIH Arrhythmia Database [90] which con- tains 48 ambulatory ECG recordings of half hour each, in order to test our methodology with this real-world problem. For this multivariate classification task, at least two channels of theECGare required. The most common channel in the database is the modified limb lead II (MLII), present in 41 recordings. This is the lead used in Chapter3for the univariate case. The second most common channel is the precordial lead V1 (See Section 3.2.2) present in 37 recordings. Thus, we constrain the database to 37 recordings in order to include these two channels for our multivariate analysis. Table4.1shows the records of the train- ing and testing datasets. Records in gray do not contain the precordial lead V1 and were extracted from the databases. These records are part of the univariate case developed in Chapter3.
As before, theECGs were down-sampled and divided into heartbeats using a fixed-length window of 170 samples locking the R-peak at sample 70. Each heartbeat was normalized with respect to the full signal to have zero mean and variance one. Figure4.10 shows the beginning of anECGof a normal subject. Both,MLII and V1 channels are represented. Blue vertical lines represent the
position of the R-peak. The N in the plot means that the heartbeats are normal. Note that in V1, the R-point causes a negative deflection because it lies in the perpendicular plane of the limb and augmented leads plane, see Fig. 3.6.
Figure 4.10: ECGtraces of the two main derivations, namelyMLIIand V1, for a normal subject. Vertical lines represent the position of the R-peak.
Figure taken from Physionet.org
In this application, we use logistic regression (Section3.4) for the learning pro- cedure.
For the classification of the heartbeats, we again utilize the Mackey-Glass non- linearity with delay feedback as the reservoir with the same configuration de- scribed in Chapter 3. We have numerically checked that, for this task, the optimal number of virtual nodes is 25 (see Fig. 3.11and its analysis).
In Chapter 3, Fig. 3.12, we have also explored the parameter space of the Mackey-Glass model finding a richer dynamics on the state matrices for ⌘ = 0.8 and = 0.5. In addition, we choose parameter p = 7 in Eq. 2.4, because for this value theNLNexhibits a short memory and a high degree of the nonlinearity, which we believe is more convenient for a classification task.
For a clinically-relevant, univariate, multiclass, heartbeat classification problem of 44 subjects from the MIT-BIH Arrhythmia Database the average performance of the classifier is shown in Table3.4. However, and in contrast to Chapter3, we construct the classifier from two channels, namely theMLIIand the V1 channels. These two channels are only available for a reduced set of patients (37 subjects), leading to a smaller usable database and a degradation in performance, due to the high variability problems of the classifier (see Fig. 3.11 and its analysis). Table4.2shows the typical measures of performance (average along all classes) of the classifier for the univariate and multivariate case for the restricted database containing both derivations. The classification for a single channel is performed using channelMLII alone (37 subjects). Then a combination of channelsMLII
and V1, denoted as MLII-V1, is used to build a multivariate classifier. It can be observed that the sensitivity increases about 9% when using two variables in comparison with the case of one variable. De Chazal et al. [73] reported an
4.3. CONCLUSIONS
improvement in accuracy of 7% in a multivariate configuration of their approach to solve this classification task.
Table 4.2: Performance of the classification of ECGs using one or two
channels.
Channel Sensitivity (%) Specificity (%) Accuracy (%) Precision (%)
MLII 76.15 97.82 98.02 78.15
MLII-V1 84.87 97.92 98.36 88.67
The accuracy is a global estimator that was not specially a↵ected by the inclusion of another channel in the inputs. However we can see that the sensitivity, i.e. the ability of the classifier to recognize the positive cases, was improved by the inclusion of channel V1.
As discussed in the analysis of Fig. 3.11, this task is particularly sensitive to the number of records in the database because the classifier su↵ers of a high variance problem. Reducing the number of records from 44 to 37 causes a degradation in the performance, e.g. sensitivity drops from 84.83 to 76.15. It is worth noting that when using the combinationMLII-V1 the sensitivity rises to comparable values of the full usable database. For future comparisons, we provide the confusion matrices of the two cases discussed in this section. Table4.3shows the confusion matrix of the classifier when usingMLIIover 35 subjects. Table4.4 shows the confusion matrix of the classifier when using the combinationMLII-V1. Note how despite the improvements shown in Table 4.2, the associated confusion matrix for MLII-V1 performs worse in the recognition of the ventricular class (V).
More tests have to be done to verify the usefulness of including more channels into the reservoir. For example, maybe including only V1 would perform better than includingMLII. We leave the completion of this task as a future work.
4.3
Conclusions
In this chapter we have numerically shown the ability of reservoir computing, based on delay-coupled systems, to perform time series prediction and classi- fication tasks following a multivariate analysis. We have concentrated on two tasks, the prediction of a chaotic time series, given by the Lorenz system, and the classification of heart beats, obtained fromECGderivations.
Table 4.3: Confusion matrix for the AAMI-classes classification problem
usingMLII(37 patients) over DS2
Predicted samples N S V F Q Known samples N 35265 1623 0 0 0 S 306 1465 26 0 0 V 38 53 3114 11 1 F 9 4 16 359 0 Q 4 0 0 2 1
Table 4.4: Confusion matrix for the AAMI-classes classification problem
usingMLII-V1 over DS2
Predicted samples N S V F Q Known samples N 35706 1182 0 0 0 S 294 1503 0 0 0 V 79 115 2991 32 0 F 8 4 12 364 0 Q 0 0 0 3 4
For the one-step prediction task of the Lorenz system, we found a significant re- duction (⇠ 3 orders of magnitude) of the normalized mean square error (NMSE) when using two variables to predict one, than when using only one variable. Moreover, we found that theNMSE remains smaller than 1% when predicting 11 steps ahead when using the two variables compared to only 7 steps for using one variable. We expect the results obtained for the well-known Lorenz system in the chaotic regime to be valid for similar multivariate dynamical systems. We have also applied the multivariate approach to the classification of heart beats. We found an improvement of 9% in sensitivity when using two channels of theECGas compared to the case when the classification was performed using a single channel.
Our results highlight that the use of more than one variable can significantly improve predictions when using reservoir computing techniques [100]. More tests with real-world data are however needed to explore the full potential of our approach.
4.3. CONCLUSIONS
It is worth noting that the fading memory present in recurrent networks resem- bles the time-delay embedding in Takens theorem. This fading memory implies that information about previous inputs is still present in the reservoir after a number of delay times ⌧. In the Lorenz time-series prediction case, adding explicitly a delayed version of the same input does not provide as much in- formation as adding the y variable. Thus, we find a significant improvement in the prediction capabilities of reservoir computing when using an additional variable.
Chapter 5
Reservoir computing using semicon-
ductor laser dynamics
The amount of information that is generated nowadays not only requires new paradigms for the solution of difficult tasks but also needs these solution to be efficient in time. In previous chapters we studied how reservoir computingis able to solve computational problems that are difficult for standard computa- tional techniques. Optical computation is an interesting approach to increase the rates of information that can be processed by a reservoir computer since it has properties such as high speed, energy efficiency and true parallelism. In this chapter we perform a thorough numerical study of the performance of a single-mode semiconductor laser subject to all-optical feedback and demon- strate how the rich dynamical properties of this delay system can be beneficially employed for processing time dependent signals. Parts of the content of this chapter is based on a collaboration published in the article:
K. Hicke, M.A. Escalona-Morán, D. Brunner, M.C. Soriano, I. Fischer and C.R. Mirasso. Information processing using transient dynamics of semiconductor lasers subject to delayed feedback. IEEE J. Selected Topics in Quantum Electronics, 19, issue 4, 1501610, (2013).
We have implemented the dynamical behavior of a semiconductor laser as the nonlinear node (NLN) of a reservoir computer for the processing of two tasks: a classification task and a time series prediction task. The implementation is depicted in Fig. 5.1. In this figure the general representation of the nonlinear node (NLN) is replaced by a semiconductor laser dynamics denoted asSL. Results presented in this chapter were obtained in close collaboration with Kon- stantin Hicke, specialist in the semiconductor laser dynamics. The numerical results were complemented by experimental results implemented in hardware, developed by Daniel Brunner in collaboration with Miguel C. Soriano, showing the robustness of the proposed scheme. This work was made under the super-
SL
Pre- processing input
layer reservoir output layer
t
Q
Figure 5.1: Schematic representation of a reservoir computer using a semi-
conductor laser (SL) as the nonlinear node (NLN) This figure is equivalent
to the general implementation of reservoir computing described in Chapter
2, Fig. 2.1.
vision of Ingo Fischer and Claudio R. Mirasso [8]. The current chapter presents only the numerical studies associated to the di↵erent tasks that were carried out by the author of this thesis.
5.1
Semiconductor laser rate equations
We consider the following model [101,102], describing the dynamics of a semi- conductor laser subject to delayed feedback. It comprises equations for the slowly varying complex electric field amplitude (in both parallel Ekand perpen-
dicular polarization direction E?, respectively) and the carrier number n in the
laser cavity:
˙Ek(t) = 12(1 + i↵) Gk(Ek,n) k Ek(t) + kEk(t ⌧ec)
+ Einj(t)ei !t+FEk, (5.1)
˙E?(t) = i ⌦E?(t) + 12(1 + i↵) G?(Ek,n) ? E?(t)
+ ?Ek(t ⌧ec) + FE?, (5.2)
˙n(t) = I(t)e en(t) Gk(Ek,n) Ek(t) 2
G?(E?,n) |E?(t)|2, (5.3)
5.1. SEMICONDUCTOR LASER RATE EQUATIONS Gk(Ek,n) = gk n(t) nT 1 + ✏ Ek(t) 2 , (5.4) G?(E?,n) = g? n(t) nT 1 + ✏ |E?(t)|2 . (5.5)
Here, ↵ is the linewidth enhancement factor, k,? are the photon decay rates of
the polarization modes, k,? the feedback rates, ⌧ecis the external cavity round-
trip time, ! denotes the detuning between the laser and the optical injection, ⌦ is the detuning betweenEk(t) and E?(t), I(t) is the time-dependent injection current, e is the elementary charge, e denotes the electron decay rate, gk,? are
the di↵erential gains, nT is the carrier number at transparency and ✏ is the gain
saturation coefficient. Ek(t) 2
and |E?(t)|2 represent the number of photons in
the parallel and perpendicular polarization direction, respectively. The output power is computed as P = [hc2↵
m/(2µg )] |E|2 where h is the Planck constant,
c the speed of light, the emission wavelength, ↵m the facet losses and µg the
group refractive index. The chosen values for these parameters used for the simulations are shown in Table5.1. the model we consider, as well as the ration of the di↵erential gains, are chosen according to [102].
The spontaneous emission noise is implemented as a complex Gaussian white noise term FEin the field equations:
FEk,? =F1+iF2, (5.6)
where the real and imaginary parts are independent random processes with
zero mean D
FEk,?(t)E= 0 (5.7)
and a variance given by D
FEk,?(t)FEk,?(t0)
E
= 2 k,? en(t) (t t0) . (5.8)
k,? are the spontaneous emission factors, describing the fraction of sponta-
neously emitted photons coupled into the respective lasing modes.
The delayed optical feedback is modeled for two configurations: polarization- maintained optical feedback (PMOF) and polarization-rotated optical feedback (PROF). ForPMOF, with the polarization direction being defined by the axis of the laser cavity yielding a higher optical gain, the optical feedback goes from the dominant mode (Ek) back to itself. For PROF, the feedback goes from the
dominant polarization mode (Ek) to the weaker polarization mode (E?). For
Table 5.1: Laser parameter values used in the numerical simulations. Parameter Value ↵ 3.0 k = ? 200 ns 1 k = ? 10 6 k =? 10 ns 1 ⌧ec 80 ns ! 0.0 ⌦ 0.0 e 1 ns 1 gk 10 5ns 1 g? 8.4 · 10 6ns 1 ✏ 10 7 1.5 µm ↵m 45 cm 1 µg 4 nT 1.8 · 108 0.4 ¯Pinj 436µW Ithr 32.0 mA
dominant field component is Ek(t) and consequently only the delay term of the
parallel component appears in the equation for E?(t). In the case ofPMOFthe
feedback rate ? is zero and in case of PROF k is zero. We also assume that
both polarization components have the same frequency, i.e. ⌦ = 0.
5.1.1
Input signal injection into the laser dynamics
For the injection of an input signal S(t) we consider two di↵erent methods: electrical injection and optical injection. In the case of electrical injection, the injection current is modulated with S(t) around a bias current, corresponding to