The spectral clustering formulation requires building a network of genes, encoding their pairwise interactions as edge weights, and analyzing the vectors and eigenvalues of a matrix derived from such a network. This procedure is well established in the literature [76] so here we limit our discussion to the main points of the algorithm and use a Markov chain perspective to help us reason further about the idiosyncrasies of the algorithm when applied to cancer expression data.
A convenient framework for understanding the spectral method is to consider the partitioning of an undirected graph πΊ =< π, πΈ > into a set of distinct clusters. Here the genes are represented as vertices π£! for π = 1 β¦ π where π is the total number of genes and network edges have weights π€!" that are non-negative symmetric (π€!" = π€!") to encode the strength of interaction between a given pair of genes. Affinities denote how likely it is for a pair of genes to belong to the same group. Here we used as affinities a modified form of the correlation coefficient π!", calculated on the gene expression vectors:
π€!" = ππ₯π Β β Β π ππ!"##$%(!!") !
!
(2.1) This is distance measure previously found to give empirical success in the clustering of gene expression data [73]. Note that high affinities correspond to pairs of genes that are likely to belong in the same group (e.g., participate in a pathway). In this paper, we ensured that the network is connected so that there is a path between any two nodes of the network. Our goal is to group genes into distinct clusters so that genes within each group are highly connected to each other, while genes in distinct clusters are dissimilar.
Spectral methods use local (pairwise) similarity (affinity) measurements between the nodes to reveal global properties of the dataset. The global properties that emerge are best understood in terms of a random walk formulation on the network [77]β[79]. The random walk is initiated by constructing a Markov transition matrix over the edge weights. Representing the matrix of affinities π€!" by π and defining the degree of a node by π! = !π€!", a Markov transition matrix π can be defined over the edge weights by
where π· is a diagonal matrix stacked with degree values π!. The transition matrix π can be used to set up a diffusion process over the network. In particular, a starting distribution π! of the
Markov chain evolves to π = π!π! after π½ iterations. As π½ Β approaches infinity, the Markov
chain can be shown to approach a stationary distribution:π!= π1! is an outer product of 1 (a
column vector of π 1s) and π (column vector of length π). It is easy to show that π is uniquely given by: π! = π!/ !π! Β and is the leading eigenvector of π: ππ = π with eigenvalue 1.
We can analyze the diffusion process analytically by using the eigenvectors and eigenvalues of M. From an eigen perspective the diffusion process can be seen as [78]:
Β π! = π + π
!!π·!.!π’!π’!!π·!!.!π! !
! (2.3)
where the eigenvalue π! = 1 is associated with stationary distribution π. The eigenvectors are arranged in decreasing order of their eigenvalues, so the second eigenvector π’! Β perturbs the stationary distribution the most as π! β₯ π! for π > 2. The matrix π’!π’!! has
elements π’!,!Γπ’!,!, which means the genes that share the same sign in π’! will have their transition probability increased, while transitions across points with different signs are decreased. A straightforward strategy for partitioning the network is to use the sign of the elements in π’! to cluster the genes into two distinct groups.
Ng et al. [80] showed how this property translates to a condition of piecewise constancy on the form of leading eigenvectors, i.e. elements of the eigenvector have approximately the same value with-in each putative cluster. Specifically, it was shown that for πΎ weakly coupled clusters, the leading πΎ eigenvectors of the transition matrix π will be roughly piecewise constant. The K-means spectral clustering method is a particular manner of employing the standard K-means algorithm on the elements of the leading πΎ eigenvectors to extract πΎ clusters
simultaneously. We follow the recipe in Ng et al. where instead of using a potentially non- symmetric matrix π, a symmetric normalized graph Laplacian πΏ = π·!!.!ππ·!!.!, whose
eigenvalues and eigenvectors are similarly related to π, is used for partitioning the graph. Spectral approaches have also some drawbacks. Their basic assumption of piecewise constancy in the form of leading eigenvectors need not hold on real data. Much work has been done to make this step robust, including the introduction of optimal cut ratios [81] and relaxations [82], [83] and highlighting the conditions under which these methods can be expected to perform well [78]. Spectral methods can be slow as they involve eigen decomposition of potentially large matrices (π(π!)). Recent attempts at addressing this issue include
implementing the algorithm in parallel [84], speeding eigen decomposition with Nystrom approximations [85], building hierarchical transition matrices [86] and embedding distortion measures for faster analysis of large-scale datasets [87].