CiteSeerX — Maximum a posteriori Maximum Entropy Order Determination

DETERMINATION

Luc Knockaert

Abstract

An instance crucial to most problems in signal processing is the selection of the order of a presupposed model. Examples are the determination of the putative number of signals present in white Gaussian noise or the number of noise-contaminated sources impinging on a passive sensor array. It is shown that Maximum a Posteriori Bayesian arguments, coupled with Maximum Entropy considerations, offer an operational and consistent model order selection scheme, competitive with the Minimum Description Length criterion.

1 INTRODUCTION

In recent years, information theoretic criteria for model order selection have been studied, es- pecially in the areas of sonar and radar array processing [1]–[5]. These criteria are principally based on the pioneering works of Akaike [6], Schwartz [7] and Rissanen [8], [9]. Since the Akaike criterion has been proved to be inconsistent [1],[3], much work has been devoted to the Schwartz- Rissanen Minimum Description Lenght (MDL) criterion. It should be noted that in Schwartz’s approach, Bayesian arguments and Maximum a Posteriori probability (MAP) considerations play a key role, while the code length mechanism advocated by Rissanen has naturally strong connec- tions with the entropy concept. This is further explained in [10], where links between Maximum Entropy (ME) and MDL have been made apparent. The aim of this paper is to show that the MAP mechanism as tutored by Van Trees [11] and ME principle as advocated by Jaynes [12]

can be combined to provide a useful criterion called Maximum a Posteriori Maximum Entropy (MAPME) criterion. The main problems which have been addressed in this context do not reside with the MAP part of the criterion, which stands firmly on its underlying Bayesian support, but with the ME part, where the constraints and Lagrange multipliers [13] have to be defined, con- structed and dealt with appropriately, taking into account all prior knowledge of the data. This

1EDICS SP 3.5

2Luc Knockaert, Dept. of Information Technology INTEC, St. Pietersnieuwstraat 41, B-9000 Gent, Belgium. Tel: +32 9 264 33 16, Fax: +32 9 264 42 99, E-mail: [email protected]

is necessary in order to avoid that the Lagrange multipliers play a role comparable to the test sizes α and thresholds in sequential Neyman-Pearson hypothesis testing [14], which in general require subjective judgments and thereby hamper the reproducibility of the results. The idea behind the MAPME construction is the consideration that a model order selection procedure generally requires the estimation of unknown parameters of two kinds. Parameters of the first kind depend a priori strongly on the model order, while parameters of the second kind (most often noise variances) do not a priori depend on the model order. E.g. if we have k sinusoids in additive white Gaussian noise with unknown variance σ², then the latter parameter is not a priori dependent on the model order k and therefore a parameter of the second kind, while the coefficients of the sinusoids themselves are certainly parameters of the first kind. Parameters of the first kind can in general only be estimated, most often by a Maximum Likelihood (ML) pro- cedure, when the model order is known. For parameters of the second kind, by definition, there exist in general estimators with useful properties - unbiasedness, consistency -, especially when the number of observations is large, independent of the model order. In this way we can consider the parameters of the second kind as asymptotically known and, by feeding this knowledge back via a global ME approach, obtain useful values for the Lagrange multipliers. The Bayes prior [15] created in this way then leads to a straightforward MAP estimation of the model order.

This paper is organized as follows. After the mathematical problem statement in Section II, we apply the MAPME criterion to the signal in white Gaussian noise, or Least Squares problem, in Section III and to the Equal Variance problem [14], [1] in Section IV. Consistency proofs, under quite general conditions, and computer simulations comparing MAPME and MDL are presented as well, illustrating the behavior of these criteria in different noise environments.

2 PROBLEM FORMULATION

Suppose we have M data vector observations xt ∈ R^N, all i.i.d. distributed with the same probability density p(x|θ), characterized by some unknown parameter vector θ ∈ Θ. The problem at hand is one of hypothesis testing i.e., given a partition of Θ in K mutually disjoint model subsets Θ_k, find a criterion to decide, solely based on the observations X = (x₁, . . . , x_M), to which set Θ_k the parameter θ belongs. In other words, one wishes to estimate the order k of the model. If the exact order k is known, an estimate of θ can in general be obtained by the

Maximum Likelihood (ML) procedure:

θˆ_k= arg max

θ∈Θkp(X|θ), (1)

where

p(X|θ) =

M

Y

t=1

p(xt|θ). (2)

Note that one of the basic limitations of the ML method [10] is precisely that the exact order has to be known, i.e. an estimate ˆk based on a ’straightforward’ extension of (1) produced by

ˆk = arg max

k p(X|ˆθ_k), (3)

does not yield in general a useful estimate of k. By contrast, the MDL estimator [7]–[9]

k = arg minˆ

k

½

− ln p(X|ˆθk) +1

2skln M

¾

, (4)

where s_kis the number of parameters needed to describe the set Θ_k, has been proved to be highly relevant [1]–[5]. Defining the indicator functions of the sets Θ_k as

Z_k(θ) = 1 if θ ∈ Θk else Z_k(θ) = 0, (5) we can interpret p(X|θ)Zk(θ) as the joint probability density p(X, k|θ) of X and k given θ, and rewrite (1) as

θˆ_k= arg max

θ p(X, k|θ). (6)

It is seen that knowledge of an a priori probability density p(θ) is sufficient to launch a Bayesian MAP procedure [11], [15], compatible with the above ML estimators, for obtaining an estimate ˆk by

k = arg maxˆ

k p(k|X) (7)

= arg max

k p(X, k)

= arg max

k

Z

p(X, k|θ)p(θ)dθ. (8)

Since p(θ) is a priori unknown, we have to estimate it, and we do so by maximizing the total entropy [12]

S = −

K

X

k=1

Z Z

p(X, k, θ) ln p(X, k, θ) dX dθ

= −

K

X

k=1

Z Z

p(X|θ)Zk(θ)p(θ) ln (p(X|θ)Zk(θ)p(θ)) dX dθ. (9)

Observing that Z_k(θ) ln Z_k(θ) = 0 ( 0 · ln 0 = 0 ), we find easily that S =

Z

p(θ) [M H(θ) − ln p(θ)] dθ, (10)

where the conditional entropy H(θ) is defined as H(θ) = − 1

M Z

p(X|θ) ln p(X|θ) dX

= − Z

p(x|θ) ln p(x|θ) dx. (11)

In general, for a ME technique to work, we must maximize the entropy S over a constraint set defined by statistical knowledge of some averages over p(θ), [10], [13] i.e.

E [g_l(θ)] = Z

p(θ)g_l(θ) dθ = ²_l for l = 1, · · · , L. (12) Note that MAP philosophy allows the ²l to be functions of the data X, but on the other hand the functions g_l(θ) and ²_l must not depend on the model order k, since this is precisely what one has to estimate. In other words the gl(θ) should be parameters of the second kind and the ²l

their estimators.

With all this in mind, ME maximization yields:

ln p(θ) = M H(θ) −

L

X

l=0

µ_lg_l(θ), (13)

where the Lagrange multipliers µ_l can themselves, by (12), be functions of the data X. Finally, the MAP probability to be maximized is simply :

p(X, k) = Z

Θk

p(X|θ)p(θ) dθ. (14)

We are now in a position to apply the above formalism to two important problems, namely, the Least Squares Problem and the Equal Variance Problem.

3 THE LEAST SQUARES PROBLEM

Least squares LS fitting problems can be conveniently described by the model

x= f (a) + σn, (15)

where x is the N −dimensional data vector (x1, . . . , x_N) and n is white N (0, I) Gaussian noise.

The unknown positive parameter σ is such that σ²represents the variance or power of the effective

noise. The modeling function f (a) expresses the dependence of the model on the model coefficient vector a ∈ A, and the problem is to decide to which one of the predefined mutually disjoint sets A_k ⊂ A it belongs. In the terms of the previous section we have

θ = (σ, a), Θ_k= [0, ∞) × Ak. (16)

Denoting Euclidian distance by |.|, we have p(x|θ) = 1

(√

2πσ)^Ne^{−2σ21 |x−f(a)|2}, (17) H(θ) = N

2 + N ln(√

2πσ), (18)

p(X|θ) = 1 (√

2πσ)^{M N}e^−2σ2M [V +|m−f(a)|²], (19) where

m= 1 M

M

X

t=1

x_t, (20)

V = 1 M

M

X

t=1

|xt|²− |m|². (21)

It should be noted that V and m are sufficient statistics, which implies that the estimation of all parameters, including the model order, have to depend exclusively on their values. By (19) it is seen that m is distributed as N (f (a), σ^{2 I}_M), and hence explicitly dependent on the model order k. The corresponding ML maximization is therefore equivalent to the LS minimization:

ˆ

a_k = arg min

a∈Ak|m − f(a)|². (22)

On the other hand V is translation invariant i.e.

V (x₁+ z, . . . , x_M + z) = V (x₁, . . . , x_M) ∀ z, (23) and therefore never dependent on the model order. In addition M V /σ² is χ² distributed with N (M − 1) degrees of freedom -here we suppose M > 1 -, and hence MV/N(M − 1) is a consistent unbiased estimator of the noise variance σ², which is therefore a parameter of the second kind.

Hence we take as the only (L = 1) nontrivial ME constraint the noise power constraint

g₁(θ) = σ² with ²₁ = M V /N (M − 1). (24) This leads to

p(θ) = C(√

2πσ)^{N M}e^−µσ2, (25)

where C is a normalizing constant. After some elementary calculations, based on the requirement that E[σ²] = ²₁, we obtain an explicit expression for the Lagrange multiplier µ :

µ = M N²

2V (1 − 1

M)(1 + 1

N M). (26)

The joint probability density p(X, k) is given by:

p(X, k) = C Z

Ak

½Z _∞

0 e^−2σ2M [V +|m−f(a)|²]e^−µσ2dσ

¾

da. (27)

The integral between the curly brackets can be explicitly calculated [16] as Z _∞

0 e^{−µσ2−α/σ2}dσ = 1 2

rπ

µe^−2√αµ α ≥ 0, µ > 0, (28) yielding

p(X, k) = C 2

rπ µ

Z

Ak

e^{−βF (a)}da, (29)

where β = M N^q(1 −_M¹)(1 + _{N M}¹ ) and

F (a) = s

1 +|f(a) − m|²

V . (30)

Since the integral in (29) is in general difficult to calculate and β is a ’huge’ parameter, we have to resort to standard asymptotic methods [17]. The function F (a) is minimized in A_k at the ML estimator ˆa_k and hence we have, in the vicinity of the minimum:

F (a) ≈ Fk+1

2(a − ˆak)^TF_k⁰⁰(a − ˆak), (31) where F_k and F_k⁰⁰ are respectively the function F and its constrained Hessian [18], which is supposed to exist in the neighborhood of the minimum ˆa_k. Hence

p(X, k) ≈ C0e^−βFk| det(Fk⁰⁰β/2π)|⁻¹². (32) Note that, if it weren’t for the determinantal factor in (32), the asymptotic maximization of p(X, k) would be equivalent with the minimization of F_k, i.e. to the maximization of the likeli- hood. As has been said before, this would make no sense, since for general order determination problems the likelihood sequence is always increasing. The role played by the determinantal factor in (32), which is a measure for the ’spread’ of the ML estimator, is therefore of crucial importance.

The maximization of p(X, k) is asymptotically equivalent to the minimization of MAPME(k) = F_k+ s_kln β

2β + 1

2β ln | det(Fk⁰⁰/2π)|, (33) where s_k is the dimension of the Hessian F_k⁰⁰. In the sequel we will suppose that the s_k form a strictly increasing sequence of numbers. Interestingly enough, it is straightforward to show that the corresponding MDL criterion, if we interpret M N as the number of observed data points, can be written as

MDL(k) = ln(F_k) + s_kln(M N )

2M N , (34)

after division by M N. Note that in the limit, for M N → ∞, the MAPME criterion is approxi- mately

MAPME(k) ≈ Fk+ s_kln(M N )

2M N . (35)

Next we prove the consistency of both criteria under quite general conditions. We need the following :

Definition: A least squares problem characterized by the function f (a) and partition sets A_k such that A1⊂ A²⊂ · · · ⊂ AK−1⊂ A^K is unisolvent if the sequence

τ_k= min

a∈A^k|f(a) − f(a⁰)|² (36)

is such that

τ₁ > τ₂ > · · · > τq = · · · = τK = 0 (37) whenever a₀ ∈ Aq.

Theorem 1: Let the unisolvent LS problem be characterized by γ_k= 1

V min

a∈Ak|f(a) − m|². (38)

Let Φ(x) be a C¹, strictly increasing and concave function defined over [0, ∞). Let vk > 0 be a given strictly increasing sequence and ω(M ) be such that ω(M ) → ∞, ω(M)/M → 0 for M → ∞.

Then the criterion which minimizes

Ξ(k) = Φ(γ_k) + v_kω(M )

M (39)

is consistent.

Proof: Without loss of generality we can suppose that Φ(0) = 0. Let q be the correct order. Then m= f (a₀) + σ

√Me, (40)

where a0 ∈ A^q and e is N (0, I).

When k < q we have

γ_k → τk/N σ² (41)

and

Ξ(k) → Φ(τk/N σ²), (42)

for M → ∞ with probability 1. This proves that in the limit Ξ(k) is strictly decreasing for k < q.

When k > q we have

γ_k≤ σ²|e|²/M V = 1

M − 1u, (43)

where u is F distributed [14] with N and N (M − 1) degrees of freedom. Utilizing Chebyshev’s inequality we obtain

P

·

Φ(γ_k) ≥ vk

ω(M ) M

¸

≤ M

v_kω(M )E [Φ(γ_k)]

≤ M

v_kω(M )Φ (E [γ_k]) (44)

≤ M

v_kω(M )Φ

µ N

N (M − 1) − 2

¶

≤ 1

v_kω(M )

M N

N (M − 1) − 2Φ⁰(0) → 0, (45) where (44) is a consequence of Jensen’s inequality [14] for concave functions. This means that for k > q the strictly increasing term v_kω(M )/M is dominating in Ξ(k) with probability 1, which completes the proof.

Note that the functions Φ(x) for the MAPME and MDL criteria are respectively √

1 + x − 1 and ¹₂ln(1 + x). By inspection it is seen that the requirements of the theorem are met in both cases with ω(M ) = ln(M ) and v_k= s_k/2N.

In order to compare both criteria we consider the simple problem

f (a) = a = (a₁, . . . , a_N), A_k = {a : al= 0 for l > k}. (46) In this context we have

F_k = s

1 + PN

j=k+1m²_j

V , (47)

MAPME(k) = F_k+ kln β 2β − k

2β ln(V F_k2π), (48)

MDL(k) = ln(F_k) + kln(M N )

2M N . (49)

A computer simulation of 100 trials with M = 2, N = 20, exact order k = q = 10, where the a_j, for j ≤ q were uniformly distributed over [−1, 1], yielded the following percentages of correct detection in terms of the signal to noise ratio SNR = −10 log10σ² :

SNR dB 10 20 30 40 50 60 70

MDL % 32 62 82 76 87 81 83

MAPME % 22 77 94 97 98 99 100

A fixed SNR = 70dB and varying data length N simulation yielded the following percentages of correct detection:

N 20 30 40 50 60 70 80

MDL % 83 87 91 92 95 97 98

MAPME % 100 100 100 100 100 100 100

This indicates that for high SNR the MAPME criterion is in general superior to the MDL criterion.

The fact that the MAPME criterion outperforms the MDL criterion may be attributed to the fact that it has a Bayesian basis and as such is likely to perform better for small sample-sizes.

4 THE EQUAL VARIANCE PROBLEM

The equal variance problem is described by the model

x= Λn, (50)

where x is the N −dimensional data vector (x¹, . . . , xN), n is white N (0, I) Gaussian noise and Λ is a diagonal matrix with positive entries θ = (σ₁, . . . , σ_N). The sets A_k, for k = 0, . . . , N − 1 are defined as

A_k= {θ : σj = σ_N for j > k}. (51)

In other words we want to decide whether the last N − k variances σ²j can be considered equal.

We have

− ln p(x|θ) = 1 2

N

X

j=1

x²_j σ_j² +

N

X

j=1

ln σ_j+N

2 ln(2π), (52)

− ln p(X|θ) = M 2

N

X

j=1

y_j σ_j² + M

N

X

j=1

ln σ_j +M N

2 ln(2π), (53)

where

y_j = 1 M

M

X

t=1

x²_t,j. (54)

The conditional entropy H(θ) is given by

H(θ) = N

2 ln(2πe) +

N

X

j=1

ln σ_j. (55)

Since the y_j are consistent and unbiased estimators of the noise powers σ_j², independently of the model order, the latter can all be treated as parameters of the second kind. Therefore we take as L = N nontrivial ME constraints :

g_j(θ) = σ²_j with ²_j = y_j for j = 1, . . . , N. (56) This leads to

p(θ) = C(√

2π)^{N M}(^YN

j=1σ_j)^Me^−P

N j=1µjσ_j²

, (57)

where C is a normalizing constant. After some calculations, based on the requirements that E[σ_j²] = y_j, we obtain explicit expressions for the Lagrange multipliers :

µ_j = M + 1

2y_j . (58)

For a given model order k we obtain p(X, k) = C

Z

e−G(σ¹, . . . , σ_k, σ_N; y)dσ₁· · · dσkdσ_N, (59) where

G(σ1, . . . , σ_k, σN; y) = ^M₂ µ

Pk

j=1yj/σ²_j +_σ¹2 N

PN j=k+1yj

¶ +

M +1 2

³Pk

j=1σ_j²/y_j+ σ²_N^PN_j=k+11/y_j^´. (60) Taking advantage of (28), the following explicit expression is readily obtained :

− ln p(X, k)

pM (M + 1) = C₀+ v u u t

N

X

j=k+1

y_j

N

X

j=k+1

1/y_j+ k(1 + γ(M )) +

1

2^qM (M + 1)

³ln(^PN_j=k+11/y_j) −^Pkj=1ln y_j^´, (61)

where

γ(M ) = ln(M + 1) − ln(π/2)

2^pM (M + 1) . (62)

Before proceeding, we define the arithmetic, geometric and harmonic means a(k), g(k) and h(k) by

a(k) = 1

N − k

N

X

j=k+1

y_j,

ln g(k) = 1 N − k

N

X

j=k+1

ln y_j, (63)

1

h(k) = 1

N − k

N

X

j=k+1

1 yj

.

Note that h(k) ≤ g(k) ≤ a(k). With this in mind, the maximization of p(X, k) is equivalent to the minimization of

MAPME(k) = (N − k)

"s a(k) h(k) − 1

#

+ kγ(M ) +

1 2√

M (M +1)[(N − k) ln g(k) + ln(N − k) − ln h(k)] . (64) In order to prove the consistency of the MAPME criterion, we need the following :

Lemma: The expression

η(k) = (N − k)

"s a(k) h(k) − 1

#

(65) is a decreasing function of k.

Proof: The function η(k) can be written as η(k) =^pξ(k) + k − N, where ξ(k) =

N

X

j=k+1

y_j

N

X

j=k+1

1

y_j. (66)

Clearly

ξ(k − 1) = ξ(k) + 1 + yk N

X

j=k+1

1 yj

+ 1 y_k

N

X

j=k+1

y_j. (67)

For all positive reals u, v, the minimum miny>0

µv y + uy

¶

= 2√

vu (68)

is obtained for y =^pv/u and hence

ξ(k) + 1 + 2^qξ(k) ≤ ξ(k − 1), (69)

which completes the proof. Note that when y_k 6= yk+1 = · · · = yN, the above reasoning also implies that η(0) ≥ · · · ≥ η(k − 1) > η(k) = · · · = η(N − 1) = 0.

Theorem 2: The MAPME criterion (64) is consistent.

Proof: Since γ(M ) → ln(M)/2M, it is sufficient to prove the consistency of the dominating term in (64), namely, in terms of the function η(k) of the Lemma, the term

ζ(k) = η(k) +k 2

ln M

M . (70)

By the Lemma the function ζ(k) is the sum of the decreasing function η(k) and an increasing linear function of k. Let q be the correct order. We shall prove that, in the limit for M → ∞, and with probability 1, the linear part is preponderant for k > q, while η(k) is preponderant for k < q.

Case k > q. By Chebyshev’s inequality we have : P

·

η(k) ≥ k 2

ln M M

¸

≤ 2M

k ln ME[η(k)]. (71)

Now

E[η(k)] = E





 v u u u t

N

X

j=k+1

y_j

N

X

j=k+1

1 y_j





− (N − k)

≤ v u u u tE





N

X

j=k+1

yj N

X

j=k+1

1 y_j



− (N − k). (72)

Since the yi/yj, for i 6= j are F distributed with equal degrees of freedom M, we obtain easily E[η(k)] ≤ N − k − 1

M − 2 (73)

and

P

·

η(k) ≥ k 2

ln M M

¸

≤ 2(N − k − 1)M

(M − 2)k ln M → 0 (74)

for M → ∞.

Case k < q. Let α be such that 0 < α < ¹₂. We have

P

·

η(k) ≤ k 2

ln M M

¸

= P

"

η(k)^−α≥ µk

2 ln M

M

¶_−α#

≤ µk

2 ln M

M

¶α

E^£η(k)^−α¤

≤ µk

2 ln M

M

¶α

E^£η(q − 1)^−α¤. (75)

Arguments of the same vein as utilized in the Lemma lead to η(q − 1) ≥

s

(yq+ yN)(1 y_q + 1

y_N) − 2

≥ µ

1 −^qyq/y_N

¶2

/^qyq/y_N. (76)

Defining κ = σq/σN 6= 1 we have

qy_q/y_N = κ√

z, (77)

where z is F distributed with equal degrees of freedom M. Hence

E^£η(q − 1)^−α¤≤ Ψ(M, κ, α), (78)

where

Ψ(M, κ, α) = 2κ^αΓ(M )/Γ(M/2)² Z _∞

0 x^{M −1+α}(1 + x²)^−M|x − κ|^−2αdx. (79) For κ 6= 1, standard asymptotic theory [17] leads to

Ψ(M, κ, α) ≈ κ^α|1 − κ|^−2α+ O µ 1

√M

¶

. (80)

By contrast, Ψ(M, 1, α) = O(M^α). This completes the proof, since P

·

η(k) ≤ k 2

ln M M

¸

≤ µk

2 ln M

M

¶α

κ^α|1 − κ|^−2α→ 0 (81)

for M → ∞.

The MDL criterion in the above context, after division by M, is given by : MDL(k) = 1

2(N − k) ln

·a(k) g(k)

¸ +k

2 ln M

M . (82)

In order to compare both criteria a computer simulation of 100 trials was performed with M = 10, N = 20, exact order k = q = 10, with σ_j = 10σ_N for j ≤ q. This yielded the following percentages of correct detection in terms of the final signal to noise ratio FSNR = −10 log10σ²_N.

FSNR dB 10 20 30 40 50 60 70

MDL % 82 78 76 75 74 75 78

MAPME % 50 66 74 84 89 89 91

It is seen that MDL behaves better than MAPME when the FSNR is low, but MAPME improves considerably with increasing FSNR, while MDL seems to remain approximately level.

Note that the MDL criterion formally transforms asymptotically into the MAPME criterion if one makes the substitution

ln(a/g) =⇒ 2(^qa/h − 1). (83)

We tested this substitution on the three examples given in [1] and obtained the same number of sources as those obtained by the MDL and Akaike criteria. It should be noted however that the calculation of harmonic means is computationally much easier than the evaluation of geometric means. Note also the interesting and easily proved inequalities

2(^qa/h − 1) ≥ ln(a/h) ≥ ln(a/g), (84)

relating aritmetic, geometric and harmonic means.

5 CONCLUDING REMARKS

Although we have addressed some key theoretical order selection problems in signal process- ing, much actually remains to be done. This includes the application of the MAPME method to the determination of the number of sinusoids in additive Gaussian noise, for which work is currently under way, and to a generalization of the Equal Variance formulation to the full eigendecomposition-based problem in array processing [5]. The main objective of this paper was to show that MDL is not the only possible way of looking at order selection, and that a judicious blend of Bayesian arguments and ME considerations can provide new insights into this crucial and often elusive problem.

References

[1] M. Wax and T. Kailath, ” Detection of signals by information theoretic criteria,” IEEE Trans. Acoust., Speech, Signal Processing,vol. 33, no. 2, pp. 387–392, Apr. 1985.

[2] M. Wax and I. Ziskind, ” Detection of the number of coherent signals by the MDL principle,” IEEE Trans. Acoust., Speech, Signal Processing,vol. 37, no. 8, pp. 1190–1196, Aug. 1989.

[3] Q. T. Zhang, K. M. Wong, P. C. Yip, and J. P. Reilly, ” Statistical analysis of the performance of information theoretic criteria in the detection of the number of signals in array processing,” IEEE Trans. Acoust., Speech, Signal Processing,vol. 37, no. 10, pp. 1557–1567, Oct. 1989.

[4] H. T. Wu, J. F. Yang, and F. K. Chen, ” Source number estimators using transformed Gerschgorin radii,” IEEE Trans. Signal Processing, vol. 43, no. 6, pp. 1325–1333, Jun. 1995.

[5] W. Xu and M. Kaveh, ” Analysis of the performance and sensitivity of eigendecomposition-based detectors,” IEEE Trans. Signal Processing, vol. 43, no.6, pp. 1413–1426, Jun. 1995.

[6] H. Akaike, ”A new look at the statistical model identification,” IEEE Trans. Atomat. Contr., vol. 19, pp.716–723, 1974.

[7] G. Schwartz, ” Estimating the dimension of a model,” Ann. Statist., vol. 6, no. 2, pp. 461–464, 1978.

[8] J. Rissanen, ”Modeling by shortest data description,” Automatica, vol. 14, pp. 465–471, 1978.

[9] — , ”Universal coding, information, prediction, and estimation,” IEEE Trans. Inform. Theory, vol.

30, no. 4, pp. 629–636, Jul. 1984.

[10] M. Feder, ” Maximum entropy as a special case of the minimum description length criterion,” IEEE Trans. Inform. Theory,vol. 32, no. 6, pp. 847–849, Nov. 1986.

[11] H. L. Van Trees, Detection, Estimation, and Modulation Theory, Part I. New York: Wiley, 1968.

[12] E.T. Jaynes, ” On the rationale of maximum entropy methods,” Proc. IEEE, vol. 70, Sep. 1982.

[13] J. E. Shore, ” On a relation between maximum likelihood classification and minimum relative-entropy classification,” IEEE Trans. Inform. Theory, vol. 30, no. 6, pp. 851–854, Nov. 1984.

[14] A. M. Mood, F. A. Graybill, and D. C. Boes, Introduction to the Theory of Statistics. Singapore:

McGraw-Hill, 1974.

[15] E. S. Keeping, Introduction to Statistical Inference. New York: Dover, 1995.

[16] I. S. Gradshteyn and I. M. Ryzhik, Table of Integrals, Series, and Products. New York: Academic, 1980.

[17] N. G. De Bruijn, Asymptotic Methods in Analysis. New York: Dover, 1981.

[18] D. M. Greig, Optimisation. London: Longman, 1980.