Independent component analysis (ICA)

A class of rapid algorithms for independent component analysis (ICA) is presented. This method utilizes multi-step past information with respect to an existing fixed-point style for increasing the non-Gaussianity. This can be viewed as the addition of a variable-size momentum term. The use of past information comes from the idea of surrogate optimization. There is little additional cost for either software design or runtime execution when past information is included. The speed of the algorithm is evaluated on both simulated and real-world data. The real-world data includes color images and electroencephalograms (EEGs), which are an important source of data on human-computer interactions. From these experiments, it is found that the method we present here, the RapidICA, performs quickly, especially for the demixing of super-Gaussian signals.

In this paper, the problem of signaling in co-located MIMO radars is considered. The usual idea of co- located MIMO radars is based on the transmission of orthogonal pulse coded signals. In this paper the approach of transmitting a set of proper pulse-trains is proposed. To separate the received pulse trains, three different estimators, including a new approach based on the theory of Independent Component Analysis (ICA) are proposed. According to this approach, an appropriate signal design method is presented, based on the separation performance of the ICA algorithms; it is shown that independent random sequences are proper signals in this sense. It is also shown that the proposed ICA-based estimator is less sensitive to Doppler effect compared to the two other estimators that consist of a set of filter-banks. So, with an equal number of test frequencies, better beam-forming features and less error in DOA estimation is gained in the ICA-based estimator. It is also shown that the computational load of the ICA- based estimator is much less than the presented maximum likelihood estimator.

Often, however, PCA may not be sufficient to separate the data in a desirable way due to more complex inter-dependences in the multivariate data (see e.g., Section 1.3.3 in Hyv¨ arinen et al. (2002) for an instructive example). This observation motivates the development of independent component analysis (ICA), formally introduced in its current form by Cardoso (1989b) and Comon (1994). ICA is a widely used unsupervised blind source separation technique that aims at decomposing an observed mixture of independent source signals. More precisely, assuming that the observed data is a linear mixture of underlying independent variables, one seeks the unmixing matrix that maximizes the independence between the signals it extracts. There has been a large amount of research on different types of ICA procedures and their interpretations, e.g., Bell and Sejnowski (1995, Infomax) who maximize the entropy, Hyv¨ arinen (1999, fastICA) maximizing the kurtosis or Belouchrani et al. (1997, SOBI) who propose to minimize time-lagged dependences, to name only some of the widespread examples.

Independent Component Analysis (ICA) is a signal processing method to extract independent sources given only observed data that are mixtures of the unknown sources. ICA was originally developed for the blind signal separation (BSS) problem [10]. Recently, there have been a considerable amount of papers presenting the applications of ICA algorithms on image data, such as [8, 11-13].

Of all the body’s organs the brain is the most mysterious. Main studies of this organ lay in the electrical activity of the firing neurons which cannot be directly investigated by any Magnetic Resonance Imaging (MRI) procedure. Analysis of the brain is now an increasingly important area of research for understanding and modeling it for medical diagnosis and treatment, especially for developing automated patient monitoring and computer-aided diagnosis. The Independent Component Analysis (ICA) approach is one exploratory method which has proven to be reasonably fit for the underlying assumption in Electroencephalogram (EEG), Event Related Potentials (ERP), Magnetoencephalography (MEG), Positron Emission Tomography (PET), functional Magnetic Resonance Imaging (fMRI) and Single Photon Emission Computed Tomography (SPECT). It is also effective in removing artifacts due to volume conduction through cerebrospinal fluid, skull, scalp and experimental imperfections.

This paper proposes a novel approach to content-based watermarking for image authentication that is based on Independent Component Analysis (ICA). In the scheme proposed here, ICA is applied to blocks of the host image and the resulting mixing matrix represents the features of the image blocks. Frobenius norm of the mixing matrix is adopted as the content-based feature. This is embedded as the watermark in a mid-frequency DCT coefficient of the block. This authentication technique is robust against incidental image processing operations, but detects malicious tampering and correctly locates the tampered regions.

The method can be further investigated in the following manner. First, it extracts certain stable patterns whose temporal trend perfectly matches with the physical phenomenon. Therefore, the individual stable oscillations (obtained as independent components from the spatio-temporal data) can be analyzed further to predict the time-series behavior of the oscillation. Second, it is very difficult to analyze the NAO in order to find the physical correlations between various modes that interact to produce the NAO phenomenon. However, ICA gives a mixing matrix that provides an indication about how the various modes interact (in a linear manner). Third, we assumed a linear mixture of various independent components. In further investigation, this assumption can be relaxed and nonlinear independent component analysis can be performed on these kind of spatio-temporal data sets in order to find even more meaningful characteristics.

An attempt has been made in this work to compare performance of Iris Recognition based on feature extraction using Gabor wavelet, Principal Component Analysis(PCA) and Independent Component Analysis(ICA).Features of input iris image are compared with that of stored in database. Experimental results show that algorithm can effectively distinguish different persons by identifying their irises based on the different distance classifiers. Gabor wavelet shows highest Recognition rate than rest of the methods. Some more demanding conditions of image capture, such as iris on- the-move, iris at-a-distance, iris off-axis are still need attention.

Traditional independent component analysis (ICA) method has been applied to deal with voltage flicker [4-5]. However, the traditional ICA uses Newton iterative method to optimize the cost function. It is easy to jump into local optimal solution and leads to reduce the accuracy of the ICA algorithm. Thus, this paper uses the kurtosis as cost function of ICA and uses artificial bee colony (ABC) algorithm to optimize the cost function to improve the accuracy of ICA algorithm.

The signal to noise ratio of an image under study is 8.5db. Principal component analysis will achieved an improve value of signal to noise ratio as 8.69db. Adaptive PCA has proven to show an enhanced value of signal to noise ratio upto 11.01db. Adaptive PCA has shown an improvement in noise reduction. Furthermore with independent component analysis with local maxima algorithm we could achieve an further enhancement value upto 15.18db of signal to noise ratio for the image under study. For various type of image format we get the different signal to noise ratio, and by comparing the signal to noise ratio and parameter table we can conclude that ICA is the best tool for the image denoising. The improvement of signal to noise ratio proves that ICA is powerful tool for denoising of an image. Some preliminary studies have been made about the effectiveness of Independent Component Analysis. So we can conclude that ICA-based methods give, at least for their application, significantly better results than PCA. The superiority of ICA over PCA is also implicit in the use of PCA as a preprocessing step.

Independent component analysis (ICA) is a well- known method of finding latent structure in data. ICA is a statistical method that expresses a set of multidimensional observations as a combination of unknown latent variables. ICA was originally developed for signal processing purposes, in particular for continuously distributed signals [7, 8]. Text documents is a very different application area in implementing ICA. However in statistical natural language processing, it has been observed that if text documents are presented in a numerical format, many numerical and computational methods can be utilized to analyze the textual data [8].

Abstract— The iris recognition is a kind of the biometrics technologies based on the physiological characteristics of human body, compared with the feature recognition based on the fingerprint, palm-print, face and sound etc, the iris has some advantages such as uniqueness, stability, high recognition rate, and non-infringing etc. The iris recognition is using independent component analysis can produce spatially global features. In this paper we use the feature extraction algorithm based on ICA for a compact iris code. And for segmentation of iris image is based on Daugman’s method using integrodifferential operator. We use independent component analysis to generate optimal basis elements which could represent iris signals efficiently. In practice the coefficient of this method are used as feature vectors. Then iris feature vectors are encoded into the iris code for storing and comparing individual's iris patterns.

The problem of finding a representation of multivariate random variables which maintains its essential distributional structure using a set of lower dimensional random variables has been of interest to researchers in statistics, signal processing and neural networks. Such representations of higher dimensional random vector using a lower dimensional vector provide a statistical frame- work to the identification and separation of the sources. Since the linear transformations of data are computationally and conceptually easier to implement, most of the methods are based on finding a linear transformation of the data. Some of the major approaches for solving this prob- lem include principal component analysis (PCA), factor analysis (FA), projection pursuit (PP) and independent component analysis (ICA). A distinguishing feature of the ICA compared with other source separation methods is that the lower dimensional random variables are extracted as independent sources in contrast to uncorrelated random variables (e.g., as in PCA). Jutten and Herault (1991) were perhaps the first to state the problem and coin the name ICA. Some of the early approaches to ICA are based on estimating the mixing matrix of the linear transformation by the maximization of the mutual information or the negentropy function (see Comon (1994) for details). Other methods for estimating the mixing matrix are based on gradient algorithms or cumulant functions which are described in detail by Hyvarinen et al. (2001), Cardoso and Souloumiac (1993) and the references therein.

This paper proposes a new image watermarking technique, which adopts Independent Component Analysis (ICA) for watermark detection and extraction process (i.e., dewatermarking). Watermark embedding is performed in the spatial domain of the original image. Watermark can be successfully detected during the Principle Component Analysis (PCA) whitening stage. A nonlinear robust batch ICA algorithm, which is able to efficiently extract various temporally correlated sources from their observed linear mixtures, is used for blind watermark extraction. The evaluations illustrate the validity and good performance of the proposed watermark detection and extraction scheme based on ICA. The accuracy of watermark extraction depends on the statistical independence between the original, key and watermark images and the temporal correlation of these sources. Experimental results demonstrate that the proposed system is robust to several important image processing attacks, including some geometrical transformations—scaling, cropping and rotation, quantization, additive noise, low pass filtering, multiple marks, and collusion. Keywords and phrases: watermarking, dewatermarking, independent component analysis (ICA).

In this paper, to improve the generality ability of the constrained complex ICA, we propose a new constrained noncircular complex fast independent component analysis (c-ncFastICA) algorithm. In c-ncFastICA, a new cost func- tion is built using the augmented Lagrangian method. The prior information is then combined into the fixed-point it- eration based on a quasi-Newton method. Stability ana- lysis shows that the optimal solution corresponds to the fixed point of c-ncFastICA.

Abstract. As one of the most prevalent SCA countermeasures, masking schemes are designed to defeat a broad range of side channel attacks. An attack vector that is suitable for low-order masking schemes is to try and directly determine the mask(s) (for each trace) by utilising the fact that often an attacker has access to several leakage points of the respectively used mask(s). Good examples for implementations of low- order masking schemes are the based on table re-computations and also the masking scheme in DPAContest V4.2. We propose a novel approach based on Independent Component Analysis (ICA) to efficiently utilise the information from several leakage points to reconstruct the respective masks (for each trace) and show it is a competitive attack vector in practice.

Our Contribution. By contrast, the denoising techniques are in general dis- cussed less, despite their importance in reducing the complexity of side-channel attacks especially for Common Criteria evaluation [18]. In this paper, we pro- pose the use of the Independent Component Analysis (ICA) [16,17,40] to denoise side-channel measurements. This technique is widely applied for Blind Source Separation (BSS) (see e.g. [36] for an application of the ICA in reducing the noise in natural images) and aims at finding a linear representation of the processed multivariate data so that the resulting components are statistically independent. To the best of our knowledge this is the first complete attempt to apply ICA as a preprocessing technique in side-channel context. Actually, in [26] Gao et al. have proposed a new profiled attack based on the ICA and they claimed that it could be used to improve the signal-to-noise (SNR) ratio, but they left this for further research. In another paper [9], Bohy et al. have also suggested a similar application but they didn’t provide a practical framework on how to efficiently apply it.

(Y. Wu & Zhang, 2011) used redundancy and relevance optimization for feature selection criteria and support vector machine for classification with 14 heart beat types achieving stable accuracy of 90 % in all the cases as compared to (Jiang et al., 2006b) work where high accuracy is achieved in some cases. (Shen, Wang, Zhu, & Zhu, 2010) also used Independent component analysis and support vector machine for multi lead ECG classification. Unlike others they segmented heartbeat into three segments, P wave , QRS interval and ST segment) separate features for each segment are extracted using ICA then combined to form classifier for 11 heart beat types. Structure of training data and amount of data is identified as possible area for performance improvement.

Therefore, signal source separation techniques that allow the recorded EEG waveforms to be unmixed are valuable for extracting and studying specific EP components such as the lambda wave. A detailed review of signal source separation methods is provided in Chapter 2. Techniques that can be used for this puipose are called independent component analysis (ICA) techniques. The goal of ICA techniques is to recover the independent source signals given only the recorded mixtures. ICA techniques are reviewed in reference [1.8]. Bell and Sejnowski [1.9] proposed a method for implementing ICA that extracts independent components by maximising the joint entropy (i.e. minimising the mutual information) of the separated components. Cardoso [1.10] proposed an approach for implementing ICA which exploits the fourth-order cumulant. The operation of the ICA algorithm of Bell and Sejnowski [1.9] (hereafter refereed to as BS_ICA) is based on a number of assumptions. These are: (i) the mixing process is linear, (ii) not more than one source signal has a Gaussian distribution, (iii) the source signals are stationary and statistically independent. When BS_ICA is applied to the EEG waveforms, the source signals are considered to be concurrent electromagnetic activities that are temporally independent of each other and that are generated by spatially fixed sources. These signals are mixed as they propagate from their sources to the electrode locations on the scalp.

It is well known that solutions to the nonlinear independent component analysis (ICA) problem are highly non-unique. In this paper we propose the “minimal nonlinear distortion” (MND) principle for tackling the ill-posedness of nonlinear ICA problems. MND prefers the nonlinear ICA solution with the estimated mixing procedure as close as possible to linear, among all possible solutions. It also helps to avoid local optima in the solutions. To achieve MND, we exploit a regularization term to minimize the mean square error between the nonlinear mixing mapping and the best-fitting linear one. The effect of MND on the inherent trivial and non-trivial indeterminacies in nonlinear ICA solutions is investigated. Moreover, we show that local MND is closely related to the smooth- ness regularizer penalizing large curvature, which provides another useful regularization condition for nonlinear ICA. Experiments on synthetic data show the usefulness of the MND principle for separating various nonlinear mixtures. Finally, as an application, we use nonlinear ICA with MND to separate daily returns of a set of stocks in Hong Kong, and the linear causal relations among them are successfully discovered. The resulting causal relations give some interesting insights into the stock market. Such a result can not be achieved by linear ICA. Simulation studies also verify that when doing causality discovery, sometimes one should not ignore the nonlinear distortion in the data generation procedure, even if it is weak.