INVOLVING THE KHATRI-RAO PRODUCT OF
SEVERAL POSITIVE MATRICES
ZEYAD ABDEL AZIZ AL ZHOUR AND ADEM KILICMAN
Received 15 February 2005; Accepted 16 October 2005
Recently, there have been many authors, who established a number of inequalities in-volving Khatri-Rao and Hadamard products of two positive matrices. In this paper, the results are established in the following three ways. First, we find generalization of the inequalities involving Khatri-Rao product using results given by Liu (1999), Mond and Peˇcari´c (1997), Cao et al. (2002), Chollet (1997), and Visick (2000). Second, we recover and develop some results of Visick. Third, the results are extended to the case of Khatri-Rao product of any finite number of matrices. These results lead to inequalities involving Hadamard product, as a special case.
Copyright © 2006 Z. A. Al Zhour and A. Kilicman. This is an open access article distrib-uted under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
1. Introduction
Consider matricesAandBof orderm×nand p×q, respectively. LetA=[Ai j] be par-titioned with Ai j of order mi×nj as the (i,j)th block submatrix and letB=[Bkl] be partitioned withBkl of order pk×ql as the (k,l)th block submatrix (m=ti=1mi, n= d
j=1nj, p= u
k=1pk, q= v
l=1ql). For simplicity, we say that AandB arecompatible partitioned ifA=[Ai j]ti,j=1 andB=[Bi j]ti,j=1 are square matrices of orderm×mand
partitioned, respectively, withAi jandBi jof ordermi×mj(m=ti=1mi=tj=1mj). Let A⊗B, A◦B,AΘB, and A∗B be the Kronecker, Hadamard, Tracy-Singh, and Khatri-Rao products, respectively, ofA and B. The definitions of the mentioned four matrix products are given by Liu in [5,6] as follows:
(i)Kronecker product
A⊗B=ai jB
i j, (1.1)
whereA=[ai j],B=[bkl] are scalar matrices of orderm×nand p×q, respec-tively,ai jBis of orderp×q, andA⊗Bof ordermp×nq;
Hindawi Publishing Corporation Journal of Inequalities and Applications Volume 2006, Article ID 80878, Pages1–21
(ii)Hadamard product
A◦B=ai jbi j
i j=B◦A, (1.2)
whereA=[ai j],B=[bi j] are scalar matrices of orderm×n,ai jbi jis a scalar, and
A◦Bis of orderm×n; (iii)Tracy-Singh product
AΘB=Ai jΘB
i j=
Ai j⊗Bkl
kl
i j, (1.3)
where A=[Ai j], B=[Bkl] are partitioned matrices of order m×n and p×
q, respectively,Ai j is of ordermi×nj,Bkl of order pk×ql,Ai j⊗Bkl of order
mipk×njql,Ai jΘBof ordermip×njq(m= t
i=1mi,n= d
j=1nj, p= u
k=1pk,
q=v
l=1ql), andAΘBof ordermp×nq; (iv)Khatri-Rao product
A∗B=Ai j⊗Bi j
i j, (1.4)
whereA=[Ai j],B=[Bi j] are partitioned matrices of orderm×nand p×q, respectively,Ai jis of ordermi×nj,Bklof orderpi×qj,Ai j⊗Bi jof ordermipi×
njqj(m= t
i=1mi,n= d
j=1nj, p= t
i=1pi, q= d
j=1qj), andA∗Bof order
M×N(M=t
i=1mipi,N=dj=1njqj).
In general,AΘB=BΘA,A⊗B=B⊗A,A∗B=B∗A, but ifA=[ai j] is a scalar matrix andB=[Bi j] is a partitioned matrix, thenA∗B=B∗A. Additionally, Liu [5] shows that the Khatri-Rao product can be viewed as a generalized Hadamard product and the Tracy-Singh product as a generalized Kronecker product, as follows:
(1) for a nonpartitioned matrixA, theirAΘBisA⊗B, that is,
AΘB=ai jΘB
i j=
ai j⊗Bkl
kl
i j=
ai jBkl
kl
i j=
ai jB
i j=A⊗B; (1.5)
(2) for nonpartitioned matricesAandBof orderm×n, theirA∗BisA◦B, that is,
A∗B=ai j⊗bi j
i j=
ai jbi j
i j=A◦B. (1.6)
The Khatri-Rao and Tracy-Singh products are related by the following relation [5,6]:
A∗B=ZT
1(AΘB)Z2, (1.7)
whereA=[Ai j] is partitioned withAi jof ordermi×njandB=[Bkl] is partitioned with
Bklof orderpk×ql(m=it=1mi,n=dj=1nj, p=uk=1pk,q=vl=1ql),Z1is anmp× r(r=t
i=1mipi) matrix of zeros and ones, andZ2 is annq×s(s=
d
of zeros and ones such thatZT1Z1=Ir,Z2TZ2=Is (Ir andIsarer×r ands×s identity matrices, resp.).
In particular, ifm=nandp=q, then there exists anmp×r(r=t
i=1mipi) matrixZ such thatZTZ=I
r(Iris anr×ridentity matrix) and
A∗B=ZT(AΘB)Z. (1.8)
Here
Z= ⎡ ⎢ ⎢ ⎣
Z1
. ..
Zt ⎤ ⎥ ⎥
⎦, (1.9)
where eachZi=[0i1···0ii−1Imi pi 0i i+1···0it]T is an real matrix of zeros and ones, and 0ik is
amipi×mipkzero matrix for anyk=i. Note also thatZiTZi=Iand
ZiTAi jΘBZj=ZiT
Ai j⊗BklklZj=Ai j⊗Bi j, i,j=1, 2,. . .,t. (1.10)
In [5–8], the authors proved a number of equalities and inequalities involving Khatri-Rao and Hadamard products of two matrices. Here we extend these results in three ways. First, we establish new attractive equalities and inequalities involving Khatri-Rao prod-uct of matrices. Second, we recover and develop some results of Visick, for example, [8, Theorem 11, page 54]. This does not follow simply from the work of Visick. Third, the results are extended to the case of Khatri-Rao products of any finite number of matrices. This result leads to inequalities involving Hadamard product, as a special case.
We use the following notations:
(i)Mm,n—the set of allm×nmatrices over the complex number fieldCand when
m=n, we writeMminstead ofMm,n;
(ii)AT,A∗,A+,A−1—the transpose, conjugate transpose, Moore-Penrose inverse,
and inverse of matrixA, respectively.
For Hermitian matricesAandB, the relationA > Bmeans thatA−B >0 is a positive definite and the relationA≥BmeansA−B≥0 is a positive semidefinite. Given a positive definite matrixA, its positive definite square root is denoted byA1/2. We use the known
fact “for positive definite matricesAandB, the relationA≥BimpliesA1/2≥B1/2” which
is called theL¨owner-Heinz theorem.
2. Some notations and preliminary results
LetAbe a positive definitem×mmatrix. Thespectral decompositionof matrixAassures that there exists a unitary matrixUsuch that
A=U∗DU=U∗diagλi
whereD=diag(λi)=diag(λ1,. . .,λm) is the diagonal matrix with diagonal entriesλi(λi are the positive eigenvalues ofA). For any real numberr,Aris defined by
Ar=U∗DrU=U∗diagλriU. (2.2)
IfA∈Mm,nis any matrix with rank (A)=s, thesingular value decompositionofAassures that there are unitary matricesU∈MmandV∈Mnsuch that
A=UV∗. (2.3)
Here=[W0 00]∈Mm,n, whereW=diag(σ1,. . .,σs)∈Msis the diagonal matrix with di-agonal entriesσi (i=1, 2,. . .,s) andσ1≥σ2≥ ··· ≥σs>0 are the singular values ofA, that is,σ1≥σ2≥ ··· ≥σs>0 are positive square roots of positive eigenvalues ofA∗Aand
AA∗. TheMoore-Penrose inverseofAis defined by
A+=V
W−1 0
0 0
U∗∈Mn,m, (2.4)
where W−1=diag(σ−1
1 ,σ−21,. . .,σ−s1)∈Ms is the diagonal matrix with diagonal entries
σ−1
i (i=1, 2,. . .,s).A+is a unique matrix which satisfies the following conditions:
AA+A=A, A+AA+=A+, AA+∗=AA+, A+A∗=A+A. (2.5)
For any compatible partitioned matricesA,B,C, andD, we will make a frequent use of the following properties of the Tracy-Singh product (see e.g., [1,3,5,10]):
(a) (AΘB)(CΘD)=(AC)Θ(BD) ifACandBDare well defined; (b) (AΘB)r=ArΘBr ifA∈M
m, B∈Mnare positive semidefinite matrices andris any real number;
(c) (AΘB)∗=A∗ΘB∗; (d) (AΘB)+=A+ΘB+.
IfA∈MmandB∈Mnare positive semidefinite matrices, then (see, [3,10]) (e)AΘB≥0;
(f)λ1(AΘB)=λ1(A)λ1(B), λmn(AΘB)=λm(A)λn(B),
whereλ1(A),λm(A) are the largest and smallest eigenvalues, respectively, of a matrixA, andλ1(B),λn(B) are the largest and smallest eigenvalues, respectively, of a matrixB.
The Khatri-Rao and Tracy-Singh products ofkmatricesAi(1≤i≤k,k≥2) will be denoted byki=1∗Ai=A1∗A2∗ ··· ∗Ak and ki=1ΘAi=A1ΘA2Θ···ΘAk, respec-tively.
Lemma2.1. LetAiandBi(1≤i≤k,k≥2)be compatible partitioned matrices. Then (i)
k
i=1
ΘAi
k
i=1
ΘBi
=
k
i=1
ΘAiBi
(2.6)
ifAiBi(1≤i≤k,k≥2)are well defined; (ii)
k
i=1
ΘAi +
= k
i=1
ΘA+
i, k=2, 3,. . .; (2.7)
(iii)
k
i=1
ΘAi ∗
= k
i=1
ΘA∗i ,
k
i=1
∗Ai ∗
= k
i=1
∗A∗i, k=2, 3,. . .; (2.8)
(iv)
k
i=1
ΘAi r
= k
i=1
ΘAr
i ifAi∈Mm(i)(1≤i≤k,k≥2) (2.9)
are positive semidefinite matrices andris any real number; (v)
k
i=1
AiΘBi
=
k
i=1 Ai
Θ
k
i=1 Bi
, k=2, 3,. . . . (2.10)
Proof. The proof is immediately derived by induction onk.
Lemma 2.2. Let Ai=[A(ghi)]∈Mm(i),n(i) (1≤i≤k, k≥2)be partitioned matrices with A(ghi) as the(g,h)th block submatrix(m=k
i=1m(i), n=
k
i=1n(i), r=
t j=1
k i=1mj(i),
s=t j=1
k
i=1nj(i), m(i)=tj=1mj(i), n(i)=tj=1nj(i)). Then there exist two real ma-tricesZ1of orderm×randZ2of ordern×ssuch thatZ1TZ1=Ir,Z2TZ2=Is(Z1,Z2are real
matrices of zeros and ones) and
k
i=1
∗Ai=Z1T
k
i=1
ΘAi
Z2, k=2, 3,. . ., (2.11)
m(i)=n(i) (1≤i≤k,k≥2), then there exists anm×rmatrixZ of zeros and ones such thatZTZ=I
r,
k
i=1
∗Ai=ZT
k
i=1
ΘAi
Z, k=2, 3,. . ., (2.12)
andZZT is anm×mdiagonal matrix of zeros and ones, so
0≤ZZT≤I
m, (2.13)
wherem=k i=1m(i).
Proof. The special case in (2.12) ofLemma 2.2is proved in [3, Corollary 2.2] and (2.13) follows immediately by the definition of matrixZ. We give proof of the general case in (2.11) ofLemma 2.2for the sake of convenience. We proceed by induction onk. Ifk=2, then (2.11) is true by (1.7). Now suppose (2.11) holds for the Khatri-Rao product ofk
matrices, that is, there exist anm×rmatrixPkrof zeros and ones and ann×smatrixRks of zeros and ones such thatPkrTPkr=Ir,RTksRks=Is, and
k
i=1
∗Ai=PkrT
k
i=1
ΘAi
Rks, k=2, 3,. . . . (2.14)
We will prove that it is true for the Khatri-Rao product ofk+ 1 matrices. Then by (1.7), there exist anm(1)r×rmatrixQ1of zeros and ones and ann(1)s×smatrixQ2of zeros
and ones such thatQT
1Q1=Ir,QT2Q2=Is, and k+1
i=1
∗Ai=A1∗
k+1
i=2
∗Ai
=QT
1
A1Θ
k+1
i=2
∗Ai
Q2=QT1
A1Θ
PT kr
k+1
i=2
ΘAi
Rks
Q2
=QT
1
Im(1)A1In(1)
Θ
PT kr
k+1
i=2
ΘAi
Rks
Q2
=QT1
Im(1)ΘPkrT
A1Θ
k+1
i=2
ΘAi
In(1)ΘRks
Q2
=QT1
Im(1)ΘPkrT
k+1
i=1
ΘAi
In(1)ΘRks
Q2.
(2.15)
LettingZ1=(Im(1)ΘPkr)Q1andZ2=(In(1)ΘRks)Q2, the inductive step is complete. Here Q1=P2r=Pr,Q1=R2s=Rs, and it is a simple matter to verify that
Z1=Im(1)ΘPkrPr=P(k+1)r, Z1T=PrT
Im(1)ΘPTkr=PT(k+1)r,
Z2=
In(1)ΘRks
Rs=R(k+1)s, Z2T=RTs
In(1)ΘRTks
=RT
(k+1)s.
Note that
ZT
1Z1=PTr
Im(1)ΘPkrT
Im(1)ΘPkr
Pr=Q1T
Im(1)ΘPkrT
Im(1)ΘPkr
Q1
=QT1
Im(1)Im(1)ΘPTkrPkr
Q1
=QT
1
Im(1)ΘIr
Q1
Im(1)ΘIr=Im(1)r
=QT
1
Im(1)r
Q1=QT1Q1=Ir.
(2.17)
Similarly, it is easy to verify thatZT
2Z2=Is.
Lemma2.3. Letαbe a nonempty subset of the set{1, 2,. . .,m}and letA∈Mmbe a positive semidefinite matrix. Then (see Chollet [4])
(i)if either−1≤r≤0or1≤r≤2, then
Ar(α)≥A(α)r, ∀α; (2.18)
(ii)if0≤r≤1, then
Ar(α)≤A(α)r, ∀α, (2.19)
whereA(α)is the principal submatrix ofAwhose entries are in the intersection of the rows and columns ofAspecified byα.
Lemma2.4. LetXj>0 (j=1, 2,. . .,k)ben×nmatrices with eigenvalues in the interval [w,W]andUj (j=1, 2,. . .,k)are r×m matrices such thatkj=1UjU∗j =I. Then (see Mond and Peˇcari´c [7])
(i)for every realp >1andp <0,
k
j=1
UjXjpU∗j ≤μ
k
j=1
UjXjU∗j p
, (2.20)
where
μ= δp−δ
(p−1)(δ−1)
p−1
p
δp−1
δp−δ p
, δ=W
w. (2.21)
While for0< p <1, the reverse inequality holds in (2.20); (ii)for every realp >1andp <0,
k
j=1
UjXpjU∗j
−
k
j=1
UjXjU∗j p
≤γ{I}, (2.22)
where
γ=Wwp−wWp
W−w + (p−1)
1
p
Wp−wP
W−w
p/(p−1)
. (2.23)
3. New applications and results
Based on the basic results inSection 2and the general connection between the Khatri-Rao and Tracy-Singh products inLemma 2.2, we generalize and derive some equalities and inequalities in works of Visick [8, Corollary 3, Theorem 4], Chollet [4], and Mond and Peˇcari´c [7] with respect to the Khatri-Rao product and extend these results to any finite number of matrices. These results lead to inequalities involving Hadamard products, as a special case.
Theorem3.1. LetAi=[A(ghi)]∈Mm(i),n(i)(1≤i≤k,k≥2)be partitioned matrices with A(ghi)as the(g,h)th block submatrix(m=k
i=1m(i),n=
k
i=1n(i))and letZ1andZ2be the
real matrices of zeros and ones that satisfy (2.11). Then
(i)there exists anm×(m−r)matrixQ(m)of zeros and ones such that the block matrix
Ω=[Z1Q(m)]is anm×mpermutation matrix.Q(m)is not unique but for any such choice ofQ(m),
Z1TQ(m)=0, QT(m)Q(m)=Im−r, Q(m)QT(m)+Z1Z1T=Im (3.1)
(ii)for anym×nmatrixL,
ZT
1LL∗Z1≥
ZT
1LZ2
ZT
1LZ2
∗≥0. (3.2)
Proof. Though the proof is quite similar to the proof of [8, Corollary 3(iii) and (vii)] for Hadamard product, we give proof for the sake of convenience.
(i) It is evident from the structure ofZ1that it may be considered as part of anm×m
permutation matrixΩ=[Z1Q(m)], whereQ(m)is anm×(m−r) matrix of zeros and ones. For example, whenk=2, thenQ(2)is not unique (see, [8, page 49]). Using the properties
of a permutation matrix together with the definition ofΩ=[Z1Q(m)], we have
Im=ΩΩT=
Z1 Q(m)
ZT
1 QT(m)
=Q(m)Q(Tm)+Z1Z1T,
Im=
Ir 0
0 Im−r
=ΩTΩ=
ZT
1 QT m
Z1 Q(m)
=
ZT
1Z1 Z1TQ(m) Q(Tm)Z1 QT(m)Q(m)
.
(3.3)
From these come the required results in (i), that is,
ZT
1Q(m)=0, Q(Tm)Q(m)=Im−r, Q(m)QT(m)+Z1Z1T=Im. (3.4)
(ii) By (2.13) ofLemma 2.2, we haveIn≥Z2Z2T≥0 and so
Z1TLL∗Z1≥Z1TLZ2Z2TL∗Z1=Z1TLZ2Z1TLZ2∗≥0. (3.5)
We now generalize [8, Theorem 4] to the case of Khatri-Rao product involving a finite
Theorem3.2. LetAi=[A(ghi)]∈Mm,n(1≤i≤k,k≥2)be partitioned matrices withA(ghi) as the(g,h)th block submatrix. LetZ1be anmk×rmatrix of zeros and ones that satisfies
(2.12) and letQ(n)be annk×(nk−s)matrix of zeros and ones that satisfies (3.1). Then
k
i=1
∗(AiA∗i)=
k
i=1
∗(Ai)
k
i=1
∗Ai ∗
+ZT1
k
i=1
ΘAi
Q(n)QT(n)
k
i=1
ΘAi ∗
Z1
=
k
i=1
∗(Ai)
k
i=1
∗Ai ∗ + ZT 1 k
i=1
ΘAi
Q(n)
ZT
1
k
i=1
ΘAi
Q(n)
∗ , (3.6) and hence k
i=1
∗AiA∗i
≥
k
i=1
∗(Ai)
k
i=1
∗Ai ∗
, k=2, 3,. . . . (3.7)
Proof. FromLemma 2.1(i) and (iii), we have
k
i=1
ΘAiA∗i
=
k
i=1
ΘAi
k
i=1
ΘAi ∗
. (3.8)
But byTheorem 3.1(i), there exist annk×s matrixZ
2 of zeros and ones that satisfies
(2.12) and an nk×(nk−s) matrixQ
(n) of zeros and ones that satisfies (3.1) such that Z2Z2T+Q(n)QT(n)=Inkand
k
i=1
ΘAiA∗i
=
k
i=1
ΘAi
Z2Z2T+Q(n)Q(Tn)
k
i=1
ΘAi ∗
=
k
i=1
ΘAi
Z2Z2T
k
i=1
ΘAi ∗
+
k
i=1
ΘAi
Q(n)QT(n)
k
i=1
ΘAi ∗
.
(3.9)
SinceAi(1≤i≤k,k≥2) are rectangular partitioned matrices of orderm×n, then due to (2.11) ofLemma 2.2there exist two real matricesZ1andZ2of zeros and ones of order mk×randnk×s, respectively, such that
k
i=1
∗Ai=ZT1
k
i=1
ΘAi
Z2, k=2, 3,. . . . (3.10)
But becauseAiA∗i (1≤i≤k, k≥2) are square matrices of orderm×m, then due to (2.12) ofLemma 2.2there exists a real matrixZ1of zeros and ones of ordermk×rsuch
that
k
i=1
∗AiA∗i
=ZT
1
k
i=1
ΘAiA∗i
Due to (3.9), (3.10), and (3.11), we have
k
i=1
∗AiA∗i
=Z1T
k
i=1
ΘAiA∗i
Z1=Z1T
k
i=1
ΘAi
Z2Z2T
k
i=1
ΘAi ∗
Z1
+Z1T
k
i=1
ΘAi
Q(n)Q(Tn)
k
i=1
ΘAi ∗ Z1 = ZT 1 k
i=1
ΘAi Z2 ZT 1 k
i=1
ΘAi
Z2
∗
+ZT
1
k
i=1
ΘAi
Q(n)Q(Tn)
k
i=1
ΘAi ∗
Z1
=
k
i=1
∗Ai
k
i=1
∗Ai ∗
+Z1T
k
i=1
ΘAi
Q(n)QT(n)
k
i=1
ΘAi ∗
Z1
=
k
i=1
∗Ai
k
i=1
∗Ai ∗
+
Z1T
k
i=1
ΘAi
Q(n)
Z1T
k
i=1
ΘAi
Q(n)
∗
.
(3.12) If we putk=2 inTheorem 3.2, we obtain the following corollary.
Corollary3.3. LetAi=[A(ghi)]∈Mm,n(1≤i≤2)be partitioned matrices withA(ghi)as the (g,h)th block submatrix. LetZ1be anm2×r matrix of zeros and ones that satisfies (1.8)
and letQ(n)be ann2×(n2−s)matrix of zeros and ones that satisfies (3.1). Then
A1A∗1 ∗A2A∗2 =
A1∗A2
A1∗A2
∗+ZT
1
A1ΘA2
Q(n)QT(n)
A1ΘA2
∗Z
1, (3.13)
and hence
A1A∗1 ∗A2A∗2 ≥
A1∗A2
A1∗A2
∗.
(3.14)
Corollary3.4. LetAi=[A(ghi)]∈Mm,n(1≤i≤k,k≥2)be partitioned matrices withA(ghi) as the(g,h)th block submatrix. LetZ1be anmk×rmatrix of zeros and ones that satisfies
(2.12) and letQ(n)be annk×(nk−s)matrix of zeros and ones that satisfies (3.1). Then the
following statements are equivalent: (i)
k
i=1
∗AiA∗i
=
k
i=1
∗Ai
k
i=1
∗Ai ∗
, k=2, 3,. . .; (3.15)
(ii)
ZT
1
k
i=1
ΘAi
(iii)
k
i=1
∗AiXi=
k
i=1
∗Ai
k
i=1
∗Xi
, forXi∈Mn,m(1≤i≤k,k≥2). (3.17)
Proof. To arrive from (i) to (ii), notice that (i) holds if and only if the last term of (3.6) is zero, which is equivalent toZ1T(
k
i=1ΘAi)Q(n)=0. To arrive from (ii) to (iii), notice
that (ii) may be rewritten asZ1T(
k
i=1ΘAi)Q(n)Q(Tn)=0. ByTheorem 3.1(i), there exist an nk×smatrixZ
2of zeros and ones that satisfies (2.12) and annk×(nk−s) matrixQ(n)of
zeros and ones that satisfies (3.1) such thatQ(n)QT(n)=Ink−Z2Z2T, this becomes
Z1T
k
i=1
ΘAi
=Z1T
k
i=1
ΘAi
Z2ZT2. (3.18)
By postmultiplying by (ki=1ΘXi)Z1for any of then×mmatricesXi(1≤i≤k), we have
Z1T
k
i=1
ΘAiXi
Z1=Z1T
k
i=1
ΘAi
Z2Z2T
k
i=1
ΘXi
Z1, (3.19)
which is (iii) by (2.11) and (2.12) ofLemma 2.2. To arrive from (iii) to (i), assume (iii) holds for alln×mmatricesXi(1≤i≤k). It must therefore be true forXi=A∗i (1≤i≤
k), which is condition (i). Hence (iii) implies (3.6) which is (i). If we putk=2 inCorollary 3.4, we obtain the following corollary.
Corollary3.5. LetAi=[A(ghi)]∈Mm,n(1≤i≤2)be partitioned matrices withA(ghi)as the (g,h)th block submatrix. LetZ1be anm2×rmatrix of zeros and ones that satisfies (1.8) and
letQ(n)be ann2×(n2−s)matrix of zeros and ones that satisfies (3.1). Then the following
statements are equivalent: (i)
A1A∗1 ∗A2A∗2 =
A1∗A2
A1∗A2
∗
; (3.20)
(ii)
ZT
1
A1ΘA2
Q(n)=0; (3.21)
(iii)
A1X1∗A2X2=
A1∗A2
X1∗X2
, forX1,X2∈Mn,m. (3.22)
Theorem3.6. LetAi≥0 (1≤i≤k,k≥2)ben×ncompatible partitioned matrices. Then (i)if either−1≤r≤0or1≤r≤2, then
k
i=1
∗Ar i≥
k
i=1
∗Ai r
(ii)if0≤r≤1, then
k
i=1
∗Ar i≤
k
i=1
∗Ai r
. (3.24)
Proof. If we puts=1, replacerby 1/randAibyAri in [3, Theorem 3.1(i)], we obtain (i). But, if we puts= −1, replacerby 1/−randAibyA−irin [3, Theorem 3.1(i)], we obtain
(ii).
Remark 3.7. It is easy to give another proof ofTheorem 3.6by replacingAbyki=1ΘAi inLemma 2.3and applying (2.12) ofLemma 2.2.
Theorem3.8. LetAi>0be compatible partitioned matrices such thatki=1ΘAi>0 (1≤
i≤k,k≥2). LetWandwbe the largest and smallest eigenvalues ofki=1ΘAi, respectively. Then
(i)for every realp >1andp <0,
k
i=1
∗Aip≤μ
k
i=1
∗Ai p
, k=2, 3,. . ., (3.25)
where
μ= δp−δ
(p−1)(δ−1)
p−1
p
δp−1
δp−δ p
, δ=W
w. (3.26)
While for every0< p <1, the reverse inequality holds in (3.25); (ii)for every realp >1andp <0,
k
i=1
∗Aip−
k
i=1
∗Ai p
≤γI, k=2, 3,. . ., (3.27)
where
γ=Wwp−wWp
W−w + (p−1)
1
p
Wp−wP
W−w
p/(p−1)
. (3.28)
While for every0< p <1, the reverse inequality holds in (3.27).
Proof. This theorem follows from [3, Theorem 3.1(ii) and (iii)]. We give proof for the sake of convenience. In (2.20) and (2.22) ofLemma 2.4, setk=1 and replaceU byZT,
U∗ byZ, and X byik=1ΘAi, whereZ, is the selection matrix of zeros and ones that satisfies (2.12). By usingLemma 2.1(iv), we establishTheorem 3.8.
From (3.25), we have the following special cases: (i) forp=2, we have
k
i=1
∗A2
i ≤
(W+w)2
4wW
k
i=1
∗Ai 2
(ii) forp= −1, we have
k
i=1
∗A−1
i ≤
(W+w)2
4wW
k
i=1
∗Ai −1
, k=2, 3,. . . . (3.30)
From (3.27), we have the following special cases: (i) forp=2, we have
k
i=1
∗A2i−
k
i=1
∗Ai 2
≤1
4(W−w)
2{I}, k=2, 3,. . .; (3.31)
(ii) forp= −1, we have
k
i=1
∗A−i1−
k
i=1
∗Ai −1
≤ √
W−√w wW
I, k=2, 3,. . . . (3.32)
4. Further developments and applications
Due toAlbert’s theoremin [2] and [9, Theorem 6.13], for a partitioned matrix [BA B∗D] with a positive (semi) definite matrixA∈Mm,
A B
B∗ D
≥0 iffD≥B∗A+B, (4.1)
for any positive semidefinite matrixD∈Mn. It is also known that if matrixAis square and nonsingular, thenA+=A−1and [A B
B∗ D]≥0 if and only ifD≥B∗A−1B.
LetZ1andZ2be the real matrices of zeros and ones of orderm×randn×s,
respec-tively, that satisfy (2.11) inLemma 2.2. Now another way to useLemma 2.2to generate inequalities involving the Khatri-Rao product is by using the following obvious inequal-ity:
TT∗=
T1 T2
T1∗ T2∗
=
T1T1∗ T1T2∗ T2T1∗ T2T2∗
≥0, (4.2)
whereT1andT2aren×landm×lmatrices, respectively. Note thatT1T1∗andT2T2∗are
positive semidefinite (positive definite) matrices for every (nonsingular) complex matri-cesT1andT2. This leads to
Z2T 0
0 ZT
1
T1T1∗ T1T2∗ T2T1∗ T2T2∗
Z2 0
0 Z1
=
Z2TT1T1∗Z2 Z2TT1T2∗Z1 ZT
1T2T1∗Z2 Z1TT2T2∗Z1
≥0, (4.3)
if and only if
ZT
1T2T2∗Z1≥
ZT
1T2T1∗Z2
ZT
2T1T1∗Z2
+ ZT
2T1T2∗Z1
Therefore (4.4) can be considered to be more general than (3.2). In order to prove this we setT1=IandT2=Lin (4.4), we have
ZT
1LL∗Z1≥
ZT
1LI∗Z2
ZT
2II∗Z2
+ ZT
2IL∗Z1
=Z1TLZ2Z2TZ2+Z2TL∗Z1 (Z2TZ2=I)
=ZT
1LZ2
ZT
1LZ2
∗.
(4.5)
Returning to (4.4) and (3.2), it can be easily seen that various other choices of the matricesT1,T2, andLare possible which lead to quite different inequalities involving
Khatri-Rao products. However, there exist some inequalities that do not seem to follow directly from (1.7) or (2.11), but follow easily from (4.4) and (3.2). Based on (4.4) and (3.2) we generalize some inequalities in works of Visick [8, Corollary 13, Remark in page 56, Theorems 11, 17, and 20] and establish some new inequalities involving Khatri-Rao products of several positive matrices.
Theorem4.1. LetA1andA2be compatible partitioned matrices. Then
A1A∗1∗A2A∗2+A2A∗2 ∗A1A∗1+A1A∗2 ∗A2A∗1 +A2A∗1 ∗A1A∗2
≥A1∗A2+A2∗A1
A1∗A2
∗
+A2∗A1
∗
. (4.6)
Proof. SetT1=IΘIandT2=A1ΘA2+A2ΘA1. Then calculations show that
T2T2∗=A1A∗1ΘA2A∗2 +A2A∗2ΘA1A∗1 +A1A∗2ΘA2A∗1+A2A∗1ΘA1A∗2, T2T1∗=A1ΘA2+A2ΘA1, T1T2∗=
A1ΘA2
∗
+A2ΘA1
∗
, T1T1∗=IΘI.
(4.7)
Substituting these into (4.4) and using (1.7), we get (4.6).
Corollary4.2. LetAi(1≤i≤2)be Hermitian compatible partitioned matrices. Then (i)
A21∗A22≥
A1∗A2
2
; (4.8)
(ii)
A2∗A−2≥A∗A−12 ifAis nonsingular; (4.9)
(iii)
I∗A2≥I∗A2. (4.10)
Proof. (i) SetA∗1 =A1andA∗2 =A2in (3.14) ofCorollary 3.3, we get (4.8).
(ii) SetA1=AandA2=A−1in (4.8), we get (4.9).
(iii) SetA1=IandA2=Ain (4.8), we get (4.10).
Corollary4.3. LetAi>0 (1≤i≤2)be compatible partitioned matrices. Then
A2 1∗A22
1/2
Proof. It follows immediately by (4.8) andL¨owner-Heinz theorem.
Theorem 4.4. Let Ai≥0 (1≤i≤k, k≥2) be compatible partitioned matrices and let
A0i =A1i/2A+1i /2=A+1i /2A1i/2(1≤i≤k). Then
2
k
i=1
∗A0i
+
A1∗
k
i=2
∗A+i
+
A+1∗
k
i=2
∗Ai
≥
A1∗
k
i=2
∗A0
i+A01∗
k
i=2
∗Ai
k
i=1
∗Ai
+
A1∗
k
i=2
∗A0
i+A01∗
k
i=2
∗Ai
.
(4.12)
Proof. Since Ai≥0 (1≤i≤k, k≥2), then A∗i =Ai. Set T1=ki=1ΘA1i/2 and T2= A11/2Θ
k
i=2ΘA+1i /2+A+11 /2Θ
k
i=2A1i/2. Since Ai1/2A1i/2=Ai,A+1i /2Ai+1/2=A+i, and A0i =
Ai1/2A+1i /2=Ai+1/2A1i/2(1≤i≤k), then calculations show that
T2T2∗=2
k
i=1
ΘA0i
+
A1Θ
k
i=2
ΘA+i
+
A+1Θ
k
i=2
ΘAi
, T1T1∗=
k
i=1
ΘAi,
T2T1∗=
A1Θ
k
i=2
ΘA0i+A0 1Θ
k
i=2
ΘAi
, T1T2∗=
A1Θ
k
i=2
ΘA0i+A0 1Θ
k
i=2
ΘAi
.
(4.13)
Substituting these into (4.4) and usingLemma 2.2, we get (4.12). If we putk=2 and replaceAibyAri(1≤i≤2) inTheorem 4.4, we obtain the following theorem.
Theorem4.5. LetA1≥0,A2≥0be compatible partitioned and letrbe any nonzero real
number such thatA01=Ar/12A1+r/2=A+1r/2Ar/12andA02=Ar/22A2+r/2=A+2r/2Ar/22. Then
2A0
1∗A02+Ar1∗A2+r+A+1r∗Ar2
≥Ar
1∗A02+A01∗Ar2
Ar
1∗Ar2
+ Ar
1∗A02+A01∗Ar2
. (4.14)
IfA1>0,A2>0 inTheorem 4.5, we obtain the following theorem.
Theorem4.6. LetA1>0,A2>0be compatible partitioned and letIbe a compatible
parti-tioned identity matrix. Then for any nonzero real numberr,
2I+Ar1∗A2−r+A−1r∗Ar2≥
Ar1∗I+I∗Ar2
Ar1∗Ar2
−1
Ar1∗I+I∗Ar2
. (4.15)
If we putr=1 andA1=A2inTheorem 4.6, we obtain the following theorem. Theorem4.7. LetA >0be compatible partitioned and letI be a compatible partitioned identity matrix. Then
In particular, ifIis a nonpartitioned identity matrix, then
2I+A∗A−1+A−1∗A≥4(I∗A)(A∗A)−1(I∗A). (4.17)
Theorem 4.8. Let A1>0 andA2>0 be compatible partitioned matrices. Then for any
nonzero real numberr
A1r∗A−2r+A1−r∗Ar2+ 2I≥
Ar/12∗A−2r/2+A−1r/2∗Ar/22
2
. (4.18)
In particular, ifA1=A2=A, Then
Ar∗A−r+A−r∗Ar+ 2I≥Ar/2∗A−r/2+A−r/2∗Ar/22. (4.19)
Proof. Since A1 >0 and A2 >0, then A∗1 =A1 and A∗2 =A2. Set L=A1r/2ΘA−2r/2+ A−1r/2ΘAr/22. Compute
ZT
1LL∗Z1=Z1TLLZ1=Z1T
Ar/2
1 ΘA−2r/2+A1−r/2ΘAr/22
Ar/2
1 ΘA−2r/2+A1−r/2ΘAr/22
Z1
=Z1T
Ar1ΘA−2r
Z1+Z1T(IΘI)Z1+Z1T(IΘI)Z1+Z1T
A−1rΘAr2
Z1
=Ar1∗A−2r+ 2I+A−1r∗Ar2.
(4.20)
Similarly,
ZT
1LZ2
ZT
1LZ2
∗ =ZT
1LZ2
2
=ZT
1
Ar/2
1 ΘA−2r/2+A1−r/2ΘAr/22
Z2
2
=Ar/12∗A2−r/2+A−1r/2∗Ar/22
2
. (4.21)
Substituting (4.20) and (4.21) into (3.2), we get (4.18).
From (4.18), we have the following special cases: (i) forr=1, we have
A1∗A−21+A1−1∗A2+ 2I≥
A11/2∗A−21/2+A−11/2∗A12/2
2
; (4.22)
(ii) forr=2, we have
A2
1∗A−22+A1−2∗A22+ 2I≥
A1∗A−21+A−11∗A2
2
From (4.19), we have the following special cases: (i) forr=1, we have
A∗A−1+A−1∗A+ 2I≥A1/2∗A−1/2+A−1/2∗A1/22; (4.24)
(ii) forr=2, we have
A2∗A−2+A−2∗A2+ 2I≥A∗A−1+A−1∗A2. (4.25)
Theorem4.9. LetA1≥0,A2≥0be compatible partitioned and letIbe a compatible
par-titioned identity matrix. Then
A2
1∞A22+ 2(A1∗A2)≥
A1∞A2
2
, (4.26)
whereA1∞A2=A1∗I+I∗A2is called the Khatri-Rao sum.
Proof. SetL=A1∇A2=A1ΘI+IΘA2(Tracy-Singh sum). SinceA1≥0 andA2≥0, then A∗1 =A1andA∗2 =A2. Calculations show that
ZT
1LL∗Z1=Z1TLLZ1=Z1T
A1ΘI+IΘA2
A1ΘI+I2ΘA2
Z1
=A2
1∗I+I∗A22+ 2
A1∗A2
=A2
1∞A22+ 2
A1∗A2
. (4.27)
Similarly,
ZT
1LZ2
ZT
1LZ2
∗=ZT
1
A1ΘI+IΘA2
Z2
ZT
1
A1ΘI+IΘA2
Z2
∗
=A1∗I+I∗A2
2
=A1∞A2
2
. (4.28)
Substituting (4.27) and (4.28) into (3.2), we get (4.26).
Theorem4.10. LetA1>0andA2>0be compatible partitioned matrices. Then for any
positive real numberr,
rA2 1∗A22
+A1A2∗A2A1+A2A1∗A1A2+1 r
A2 2∗A21
≥rA1∗A2
2
+A1∗A2
A2∗A1
+A2∗A1
A1∗A2
+1
r
A2∗A1
2 .
(4.29)
Proof. SetL=ε1A1ΘA2+ε2A2ΘA1, whereε1andε2are both positive. SinceA1>0 and A2>0, thenA∗1 =A1andA∗2 =A2. Compute
ZT
1LL∗Z1=ZT1LLZ1=Z1T
ε1A1ΘA2+ε2A2ΘA1
ε1A1ΘA2+ε2A2ΘA1
Z1
=ZT
1
ε2 1
A2 1ΘA22
+ε1ε2
A1A2ΘA2A1
+ε1ε2
A2A1ΘA1A2
+ε2
2
A2 2ΘA21
Z1
=ε21
A21∗A22
+ε1ε2
A1A2∗A2A1
+ε1ε2
A2A1∗A1A2
+ε22
A22∗A21
.
Similarly,
ZT
1LZ2
ZT
1LZ2
∗=ZT
1
ε1A1ΘA2+ε2A2ΘA1
Z2 ZT 1
ε1A1ΘA2+ε2A2ΘA1
Z2
∗
=ZT
1
ε1A1ΘA2+ε2A2ΘA1
Z2 ZT 1
ε1A1ΘA2+ε2A2ΘA1
Z2
=ε1A1∗A2+ε2A2∗A1
2
=ε2 1
A1∗A2
2
+ε1ε2
A1∗A2
A2∗A1
+ε1ε2
A2∗A1
A1∗A2
+ε22
A2∗A1
2 .
(4.31)
Substituting (4.30) and (4.31) into (3.2), we have
ε21
A21∗A22
+ε1ε2
A1A2∗A2A1
+ε1ε2
A2A1∗A1A2
+ε22
A22∗A21
≥ε2 1
A∗1A2
2
+ε1ε2
A1∗A2
A2∗A1
+ε1ε2
A2∗A1
A1∗A2
+ε2
2
A∗2A1
2 .
(4.32)
Setr=ε1/ε2, we get (4.29).
Remark 4.11. LetAi (1≤i≤k, k≥2) be compatible partitioned matrices. Then (3.7) can be proved by settingT1=
k
i=1ΘIandT2=
k
i=1ΘAi. Calculations show that
T2T2∗=
k
i=1
ΘAiA∗i , T2T1∗=
k
i=1
ΘAi, T1T2∗=
k
i=1
ΘAi ∗
, T1T1∗=
k
i=1
ΘI.
(4.33)
Substituting these into (4.4) and using (2.11), we get (3.7).
Remark 4.12. LetAi(1≤i≤2) be compatible partitioned matrices. Then (3.14) can be proved by puttingk=2 inRemark 4.11.
Remark 4.13. All results obtained in Sections3 and 4are quite general. These results lead to inequalities involving Hadamard product, as a special case, for nonpartitioned matricesAi (i=1, 2,. . .,k, k≥2) with the Hadamard product and Kronecker product replacing the Khatri-Rao product and Tracy-Singh product, respectively.
Now we utilize the commutativity of the Hadamard product to develop, for instance, (3.7) ofTheorem 3.2. This result leads to the following inequality involving Hadamard product, as a special case:
k
i=1
◦AiA∗i
≥
k
i=1
◦Ai
k
i=1
◦Ai ∗
. (4.34)
It is possible to develop (4.34) in a different direction from (3.6). For example, Visick [8, Theorem 11, page 54] proved that ifA1,A2∈Mm,nands∈[−1, 1], then
A1A∗1 ◦A2A∗2 +s
A1A∗2◦A2A∗1
≥(1 +s)A1◦A2
A1◦A2
We will extend this inequality to the case of products involving any finite number of matrices.
If the Tracy-Singh and Khatri-Rao products are replaced by the Kronecker and Hada-mard products inLemma 2.2, respectively, we obtain the following corollary.
Corollary4.14. LetAi∈Mm,n(1≤i≤k,k≥2). Then
k
i=1
◦Ai=PkmT
k
i=1
⊗Ai
Pkn, (4.36)
wherePkm=(E(11m)0(m)··· 0(m)E (m)
22 0(m)···0(m)···0(m)···0(m)E (m)
mm)T is of ordermk×m,0(m)is an
m×mmatrix with all entries equal to zero, andEi j(m)is anm×mmatrix of zeros except for a one in the(i,j)th position.
Theorem4.15. LetAi∈Mm,n(1≤i≤k, k≥2). Then for any real scalars α1,α2,. . .,αk which are not all zero,
α2
1+···+α2k k
i=1
◦AiA∗i
+ k−1
r=1 μr
k
w=1
◦AwA∗(w+r)
≥α1+···+αk 2
k
i=1
◦Ai
k
i=1
◦Ai ∗
,
(4.37)
whereμr= k
w=1αwα(w+r)andw+r≡(w+r)modkwith1≤(w+r)≤k. Proof. Let
L=α1A1⊗A2⊗ ··· ⊗Ak+α2A2⊗ ··· ⊗Ak⊗A1+···+αkAk⊗A1⊗ ··· ⊗Ak−1,
(4.38)
whereAi∈Mm,n(1≤i≤k,k≥2) andα1,α2,. . .,αkare real scalars which are not all zero. Taking indices “ modk,”Lemma 2.1(i), (iii) (by setting⊗instead ofΘ) give
LL∗= k
i=1
αiAi⊗Ai+1⊗ ··· ⊗Ai−1
k
i=1
αiA∗i ⊗A∗i+1⊗ ··· ⊗A∗i−1
=α2 1
A1A∗1⊗ ··· ⊗AkA∗k
+···+α2
k
AkA∗k ⊗AA1∗⊗ ··· ⊗AkA∗k−1
+
i=j
αiαj
AiA∗j ⊗Aj+1A∗j+1⊗ ··· ⊗Aj−1A∗j−1
.
(4.39)
Now the application of (4.36) and the commutativity of the Hadamard product yield
PTkmLL∗Pkm=α21+···+α2k k
i=1
◦AiA∗i
+ k−1
r=1 μr
k
w=1
◦AwA∗(w+r)
, (4.40)
Also by (4.36) and the commutativity of the Hadamard product, we obtain
PkmT LPkn
=PkmT α1
A1⊗A2⊗ ··· ⊗Ak
+α2
A2⊗ ··· ⊗Ak⊗A1
+···+αk
Ak⊗A1⊗ ··· ⊗Ak−1
Pkn
=α1PTkm
A1⊗A2⊗ ··· ⊗AkPkn+α2PTkm(A2⊗ ··· ⊗Ak⊗A1
Pkn
+···+αkPTkm
Ak⊗A1⊗ ··· ⊗Ak−1
Pkn
=α1
A1◦A2◦ ··· ◦Ak
+α2
A2◦ ··· ◦Ak◦A1
+···+αkAk◦A1◦ ··· ◦Ak−1
=α1+···+αk k
i=1
◦Ai
,
PkmT LPkn ∗
=α1+···+αk k
i=1
◦Ai ∗
.
(4.41)
Now
PT kmLPkn
PT kmLPkn
∗=α
1+···+αk 2
k
i=1
◦Ai
k
i=1
◦Ai ∗
. (4.42)
SincePkmT LL∗Pkm≥(PkmT LPkn)(PkmT LPkn)∗ by (3.2) and from (4.40) and (4.42), we get
(4.37).
Now, we examine some special cases briefly.
In order to see that (4.37) really is an extension in (4.34), it is sufficient to setα1=1
andα2= ··· =αk=0. Thus we recover the result of Visick in (4.35) which we mentioned before the statement ofCorollary 4.14. Letk=2, thenμ1=
2
w=1αwα(w+1)withw+ 1≡ (w+ 1)mod 2, that is,μ1=2α1α2. ThenTheorem 4.15asserts that
α21+α22
A1A∗1 ◦A2A∗2
+ 2α1α2
A1A∗2◦A2A∗1
≥α1+α2
2 A1◦A2
A1◦A2
∗
.
(4.43)
Simplification gives
A1A∗1 ◦A2A∗2 +s
A1A∗2◦A2A∗1
≥(1 +s)A1◦A2
A1◦A2
∗
(4.44)
for anys∈[−1, 1], just as we wanted. Finally, we present an attractive inequality using three matrices. Letk=3,α1=1,α2=α3= −1/2.Theorem 4.15asserts that
A1A∗1 ◦A2A∗2 ◦A3A∗3 ≥
1 2
A1A∗2◦A2A∗3◦A3A∗1+A2A∗1 ◦A3A∗2 ◦A1A∗3
5. Acknowledgments
The authors would like to thank the referees for their valuable comments and sugges-tions, including the simplified proof ofTheorem 3.6and some statements. The present research has been partially supported by University Putra Malaysia (UPM) under the Grant IRPA09-02-04-0259-EA001.
References
[1] Z. A. Al Zhour and A. Kilicman,New Holder-type inequalities for the Tracy-Singh and Khatri-Rao products of positive matrices, Proceedings of the International Conference on Mathematics, Statistics and Their Applications, vol. 1, North Sumatera, 2005, pp. 1–7.
[2] A. Albert,Conditions for positive and nonnegative definiteness in terms of pseudoinverses, SIAM Journal on Applied Mathematics17(1969), no. 2, 434–440.
[3] C.-G. Cao, X. Zhang, and Z.-P. Yang,Some inequalities for the Khatri-Rao product of matrices, Electronic Journal of Linear Algebra9(2002), 276–281.
[4] J. Chollet,Some inequalities for principal submatrices, The American Mathematical Monthly104 (1997), no. 7, 609–617.
[5] S. Liu,Matrix results on the Khatri-Rao and Tracy-Singh products, Linear Algebra and Its Appli-cations289(1999), no. 1–3, 267–277.
[6] ,Several inequalities involving Khatri-Rao products of positive semidefinite matrices, Lin-ear Algebra and Its Applications354(2002), no. 1–3, 175–186.
[7] B. Mond and J. E. Peˇcari´c,Matrix inequalities for convex functions, Journal of Mathematical Anal-ysis and Applications209(1997), no. 1, 147–153.
[8] G. Visick,A quantitative version of the observation that the Hadamard product is a principal sub-matrix of the Kronecker product, Linear Algebra and Its Applications304(2000), no. 1–3, 45–68. [9] F. Zhang,Matrix Theory. Basic Results and Techniques, Universitext, Springer, New York, 1999. [10] X. Zhang, Z.-P. Yang, and C.-G. Cao,Inequalities involving Khatri-Rao products of positive
semi-definite matrices, Applied Mathematics E-Notes2(2002), 117–124.
Zeyad Abdel Aziz Al Zhour: Department of Mathematics and Institute for Mathematical Research, University Putra Malaysia (UPM), 43400 Serdang, Selangor, Malaysia
E-mail address:[email protected]
Adem Kilicman: Department of Mathematics and Institute for Mathematical Research, University Putra Malaysia (UPM), 43400 Serdang, Selangor, Malaysia