Discussion

3.9.1

Comparison to Compressive MUSIC

An algorithm similar to SA-MUSIC named “compressive MUSIC” (CS-MUSIC) has been independently proposed by Kim et al. [81]. Although the main ideas underlying the SA-MUSIC and CS-MUSIC algorithms are similar, in fact, the two studies differ in the following significant ways.

First, the algorithms considered are different in several respects as follow: 1. The forward greedy algorithms for partial support recovery are dif- ferent. The criteria for the update of OSMP in SA-MUSIC and the update in CS-MUSIC [81] maximize ∥(P_P⊥

R(AJ )Sb)ak∥2/∥P

⊥

∥(PP_{R(AJ )}⊥ Sb)ak∥2, respectively. The OSMP terms introduce the nor-

malization of P_R(A⊥

J)ak by its ℓ2 norm, which is missing in the greedy

algorithm in CS-MUSIC [81]. Owing to this normalization, the OSMP criterion uses a valid metric between two subspaces, while the greedy algorithm in CS-MUSIC [81] does not.

2. Given partial support J1 ⊂ J0, like SA-MUSIC, CS-MUSIC also con-

structs an augmented subspace eS = bS + R(AJ1). However, the crite-

ria that determine the remaining support elements are different. SA- MUSIC and CS-MUSIC maximize∥P_Seak∥2/∥ak∥2 and∥P_Seak∥2, respec-

tively. Again, the difference in the normalization implies that the SA- MUSIC criterion is based on the subspace metric while that of CS- MUSIC is not.11

3. SA-MUSIC and CS-MUSIC differ in the estimation schemes of the sig- nal subspace. In an ideal case where XJ0

0 has full column rank (hence

N ≤ s) and Y is noise-free, a perfect signal subspace estimate is trivially computed as the range space of Y . Otherwise, a signal subspace esti- mate can be computed by a truncated SVD. However, to get a reliable signal subspace estimate, it is important to determine the dimension of the estimate carefully. We propose and analyze an algorithm for the signal subspace estimation in this perspective. Such an estimation scheme is missing in [81], which focused more on the scenario where N < s.

Second, the analyses in the two studies are fundamentally different. The analysis of Kim et al. [81] depends heavily on the assumption that A is an i.i.d. Gaussian matrix and the size of the problem goes to infinity satisfying certain scaling laws. The authors showed that under certain conditions, the probability of failure of the support recovery converges to 0 in their “large system model”. However, since no convergence rate is shown, the analy- sis provides no guarantee on any finite dimensional problem. In contrast, the guarantees in this cahpter are based on the weak-1 RIP and are non- asymptotic. Our guarantees provide explicit formulae for the required m as functions of s and n, for various sensing matrices A, including i.i.d. Gaussian

11_{Unlike the CS-MUSIC algorithm, which must always use finite data, the theoretical}

analysis in [81] is not affected by this issue, because in the large system model∥ak∥2= 1,

and random partial Fourier, whereas the analysis in [81] only applies to an i.i.d. Gaussian A.

Finally, the comparison to MUSIC [26], which is the most relevant previous work, is missing in the numerical results of [81]. In fact, in the regime where CS-MUSIC dominates other methods in [81, Fig. 5], CS-MUSIC coincides with MUSIC [26] since XJ0

0 has full row rank. However, this is not shown.

Including MUSIC in the comparison would reveal that CS-MUSIC has only a marginal advantage over MUSIC in the scenario studied there. Different scenarios would have to be studied to better characterize CS-MUSIC. In contrast, we studied the cases where MUSIC is not successful due to either rank defect and/or ill-conditioning. SA-MUSIC improves on MUSIC in the sense that SA-MUSIC performs well in such settings which are unfavorable to MUSIC.

3.9.2

Comparison to the Guarantee of M-BP with the

Multichannel Model

Various practical algorithms including p-SOMP, p-thresholding, and M-BP, have been analyzed under the multichannel model [62, 80]. Although it is restricted to the noiseless case, the average case guarantee of M-BP with the multichannel model [80] has been shown to be better than the other guarantees of the same kind for other algorithms. Therefore, we compare the guarantees of SA-MUSIC algorithms to that of M-BP.

For this comparison, we too assume that the snapshots are noise-free (i.e., Y = AJ0X

J0

0 ). Nevertheless, the guarantee of SA-MUSIC algorithms in this

chapter is restricted neither to the noiseless case nor to the multichannel model.

In the noiseless case, the signal subspace estimation is perfect bS = S , R(AJ0X

J0

0 ), with r , dim( bS) = rank(X

J0

0 ). If N ≥ s, where s is the spar-

sity level, then XJ0

0 following the multichannel model has full row rank with

probability 1. In the full row rank case, any SA-MUSIC algorithm reduces to MUSIC and provides the best possible guarantee with the minimal re- quirement αweak_s+1 (A; J0) > 0, which reduces to m > s for certain matrices

such as i.i.d. Gaussian A. This completes the comparison in the case N ≥ s. Therefore, to compare the performance of SA-MUSIC and M-BP in the rank-

defective case, we assume that N < s. The rank of XJ0

0 is then determined

by the number of snapshots (i.e., rank(XJ0

0 ) = N ); hence, r = N .

Previous work [80, Theor. 4.4] showed that M-BP is guaranteed with prob- ability 1− ϵ if A satisfies δ_s+1weak(A; J0) < δ (3.9.1) for δ satisfying ( δ 1− δ )₋₂ + 2 ln ( δ 1− δ ) ≥ 2 ln(n/ϵ) N + 1. (3.9.2) 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 r/s δ M−BP SA−MUSIC+OSMP MUSIC (a) 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 r/s δ M−BP SA−MUSIC+OSMP MUSIC (b)

Figure 3.10: Required weak-1 RIC for the guarantees of M-BP (average case analysis with the multichannel model with error probability ϵ = 10−3), and SA-MUSIC (worst case analysis) for the noiseless case (a)

n = 128, s = 8. (b) n = 1024, s = 64.

On the other hand, SA-MUSIC+OSMP is guaranteed by the weak-1 RIP and the row-nondegeneracy condition. In particular, when XJ0

0 follows the

multichannel model, it is row-nondegenerate with probability 1. There- fore, we need to only compare the admissible δ given by (3.6.17) for SA- MUSIC+OSMP to the one given by (3.9.2) for M-BP. Fig. 3.10 displays this comparison.

For both algorithms, as r/s increases, δ required for the guarantee in- creases; hence, the guarantee is obtained subject to a milder condition. Fig. 3.10(a) shows that the guarantee of SA-MUSIC+OSMP is satisfied by a larger δ; hence, the guarantee requires reduced oversampling factor m/s compared to M-BP when the problem is small (n = 128). Fig. 3.10(b) shows

that SA-MUSIC+OSMP provides a better guarantee (larger δ) than M-BP in the regime r/s≥ 0.6 when n = 1024.

The theoretical guarantee not withstanding, in our simulations, the recov- ery rate of SA-MUSIC+OSMP was always higher than that of M-BP, and often substantially so.

3.9.3

Comparison to the Analysis of Group LASSO in High

Dimension

The guarantee of Group LASSO by Obozinski et al. [66] is quite tight and, in particular, achieves the optimal guarantee by the minimal requirement (m > s) for certain scenarios. However, their guarantee is asymptotic and only applies to Gaussian A. In contrast, although our guarantee of SA- MUSIC+SS-OSMP is not as tight as that of Group LASSO [66], the guaran- tee is non-asymptotic (i.e., valid for any finite problem), and applies to wider class of matrices that arise in practical applications, including the partial Fourier case.

3.9.4

Comparison to Compressed Sensing with Block Sparsity

The joint sparse recovery problem can be cast as a special case of compressed sensing with block sparsity [27]. The block structure in the sparsity pattern in the latter problem has been exploited to improve the performance of sparse recovery (cf. [27,102,124,127]) over the unstructured original problem. How- ever, the reduction of the joint sparsity problem to the block sparsity problem results in a special case where the sensing matrix is block diagonal with re- peated blocks. Therefore, the existing analysis [127] of the block sparsity problem, which did not cover this special case, does not apply to the joint sparsity problem.