4.4 Experimental results
4.4.3 Nonlinear channel equalization
In the digital communication, channels usually distort the transmitted symbols in var- ious ways causing intersymbol interference. Therefore, at the receiver end signal pro- cessing called equalization is performed to recover the transmitted symbol. Since sym- bols are represented as complex numbers in quadrature amplitude modulation (QAM), CVNNs can serve as efficient equalizers at the receiver end. In this section, we evaluate and compare CFLN against other CVNNs for a well known complex nonminimum- phase channel model introduced by Cha and Kassam (12). The nonlinear channel is modeled as a cascade of linear filter followed by a nonlinear element as given below
xn= on+ 0.1o2n+ 0.05o3n+ vn, (4.13)
on= w0sn+ w1sn−1+ w2sn−2, (4.14)
where sn is the transmitted symbol at time step n, on the FIR filter output, xn the
Table 4.5: Performance comparison among CRBF, C-ELM, FC-RBF, FLANN, RFLN, and the proposed CFLN for nonlinear channel equalization.
Network Connection Training (MSE) Testing (MSE)
CRBF 62 0.5630 0.5972 C-ELM 62 0.5720 0.5772 FC-CRBF 62 0.3700 0.4142 FLANN 19 0.3007 0.3010 RFLN 31 0.1314 0.1367 CFLN 10 0.1378 0.1419
SNR level). The FIR filter coefficients are as follows
w0= 0.34− 0.27i, w1 = 0.87 + 0.43i, and w2= 0.34− 0.21i
The equalizer was set to have three consecutive received symbols as its input, as was done in (62). It estimated the transmitted symbol sn−τ at time n for a delay of τ = 1.
A 4-QAM random symbol sequence was passed through the channel, where the real and imaginary parts of the symbols were taken to be 0.7. The training and testing set were constructed from distorted symbols at 20dB SNR, having 5000 and 10000 samples, respectively.
Training by OLS design method yielded a fourth order polynomial, where the CFLN had nine monomials. The monomials in order of their selection during the pruning phase (see Algorithm 1) were xn, xn−1, x2n, xn−2, x2n−1, xn3xn−1,x3n, x4n, and x2nxn−1. When the
real and imaginary parts were separated and the same OLS design process was applied, a total of 30 monomials were selected by the RFLN resulting a total of 62 real-valued connection parameters in it.
Table 4.5 compares the performance of CFLN against some recently proposed complex-valued multilayer networks as well as with RFLN and FLANN in terms of connection parameters, and MSE on training and testing set. The results for multi- layer CVNNs are taken from (62). It can be observed that our proposed CFLN have the least connection parameters. Although RFLN exhibited slightly lower MSE than
4.5 Conclusions
CFLN, it was found that CFLN could achieve zero symbol error rate (SER) at 20dB SNR level, whereas the SER for RFLN was 7× 10−4. Since the purpose of channel
equalization is to achieve lower SER, our result indicates that CFLN could generalize phase better than the RFLN. Furthermore, even though FLNs are single layer networks, their design by OLS method could achieve better performance than the multilayer net- works. The superiority of FLN over MLP in nonlinear channel equalization problem was also reported in (56). In order to see the effect of noise level, we evaluated SER for different SNR. Figure 4.4 compares the CFLN with other networks in terms of SER for different SNR levels in dB. It is clearly observed that the proposed CFLN exhibits superior performance for all most all the SNR levels.
4 6 8 10 12 14 16 −3 −2.5 −2 −1.5 −1 −0.5 SNR(dB) log 10 (SER) CRBF CMRAN CELM FC−RBF RFLN CFLN
Figure 4.4: Symbol error rates at different SNR levels.
4.5
Conclusions
A new approach called CFLN is proposed in this chapter for solving complex-valued function approximation problems. The network has a single-layered structure with an enhanced input layer, where nonlinearity is imposed as a functional expansion by
multivariate polynomial. Despite the large number of total monomials in a polynomial, only the significant monomials along with their connection parameters are selected by the OLS method. Some favorable advantages of CFLNs include: no requirement of activation functions in complex domain, existence of a single minimum of the MSE cost function, and availability of very fast model selection and learning algorithm such as OLS.
Experimental studies with a synthetic function approximation, real-world wind pre- diction and channel equalization problem have demonstrated the ability of our proposed OLS based CFLN approach to select near minimal set of monomials. Moreover, ap- proximation ability was observed to be superior compared to other CVNNs. It is also shown that FLN design in complex domain is much more advantageous than in real domain for complex-valued function approximation problems.
Investigation of any suitable local pruning strategy (e.g., predicting only potential monomials before the OLS is invoked to avoid generating all N +r−1r monomials) dur- ing the constructive phase of our method can be a very useful and interesting study as a future work. Although multivariate polynomial provides linearly independent ba- sis in the monomial space, a better basis might be orthogonal polynomials. Future study could also investigate the CFLN design taking the advantage of the orthogonal polynomials.
Chapter 5
Single-Layered CVNNs for
Classification Tasks
5.1
Introduction
Pattern classification is an important task in machine learning. The goal is to categorize raw data into groups or classes. For example, in the handwritten character recognition task, a machine has to identify each character from the raw data coming from an image of a handwritten document. The first step to solve this problem is to construct a good feature vector, say x = (x1, x2, . . . , xm)T, that can provide discriminating information.
If the patterns belong to one of C classes, the learning process needs to construct a total of C discriminant functions, gi(x), 1≤ i ≤ C, representing each class. Comparing the
discriminating scores, a machine answers the class as the one with the highest score. Neural networks are one of the most widely used pattern classification tools, where each output unit represents a discriminating function. Although the feature vectors representing patterns can be real- or complex-valued, we encounter real-valued patterns most often. Because the real number set is a subset of complex number set, complex- valued neural networks (CVNNs) can be used to classify the real-valued patterns.
In Chapter 3, we have presented Wirtinger calculus based derivation of learning algorithms for feedforward CVNNs, namely, the gradient descent and LM algorithm. The main advantage of the Wirtinger calculus is a simple and easier derivation directly in the complex domain. However, it should be noted that the same result could be reached had we followed the tedious approach of formulating learning algorithms using
real derivatives. Since the LM algorithm is especially useful in function approximation problems for high accuracy (4), we will not consider it for classification tasks in this chapter. Only gradient descent algorithm will be applied.
Chapter 4 has taken a different approach of constructing CFLN which is a single- layered network with enhanced input layer. Such network is applicable to function approximation problems since activation function is not included in the network. Most importantly, target applications were modeling complex-valued data from the viewpoint of function approximations.
Usefulness of CVNNs, however, is not limited to processing complex-valued infor- mation only. Many studies (1, 43, 47, 50) have suggested that CVNNs are also useful in processing information presented in real domain, specifically, in real-valued classi- fication tasks. Here, processing real-valued information by CVNNs means that neu- ral networks are allowed to have complex-valued parameters (connection weights and biases), in contrast to real-valued neural networks (RVNNs) which allow real-valued parameters only. It might also be useful to represent real-valued information by com- plex numbers in some meaningful ways such as phasors, and then let a CVNN process the information. In this chapter, we discuss how CVNNs can be applied to the real- valued classification problems including our newly proposed method. This chapter also presents an empirical study on the real-world benchmarking classification problems and a performance enhancement by ensemble techniques.
First, we consider a single complex-valued neuron (CVN) as a binary (two-class) classifier. We discuss several approaches of using a CVN as a discriminant function proposed by other researchers, along with their shortcomings in Section 5.2. To mini- mize those shortcomings, we modify CVN model by incorporating two new activation functions. The model and its learning algorithm are explained in Section5.3. Classifi- cation ability of our modified CVN model on several Boolean problems is discussed in section 5.4. Since the CVN model can deal with two-class problems only, we consider single-layered CVNNs in order to solve multiclass problems. A single-layered CVNN consists of multiple CVNs where each CVN represents a discriminant function. That is, the class is determined in a winner-takes-all manner. Section 5.5 presents experimental studies of single-layered CVNNs over several benchmark problems. In order to enhance the performance we consider ensemble approach in Section 5.6. Finally, we summarize this chapter in Section 5.7.