Contents lists available atScienceDirect
Linear Algebra and its Applications
j o u r n a l h o m e p a g e : w w w . e l s e v i e r . c o m / l o c a t e / l a a
An extrapolated TSVD method for linear discrete ill-posed
problems with Kronecker structure
A. Bouhamidi
a, K. Jbilou
a, L. Reichel
b,∗, H. Sadok
aaLaboratoire de Mathématiques Pures et Appliquées, Université du Littoral, Centre Universtaire de la Mi-Voix, Batiment H. Poincarré, 50 Rue F. Buisson, BP 699, 62228 Calais Cedex, France
bDepartment of Mathematical Sciences, Kent State University, Kent, OH 44242, USA
A R T I C L E I N F O A B S T R A C T Article history:
Received 25 April 2010 Accepted 28 May 2010 Available online 17 July 2010 Submitted by B.N. Datta Keywords:
Ill-posed problem Kronecker product
Truncated singular value decomposition Vector extrapolation
Truncation criterion
This paper describes a new numerical method for the solution of large linear discrete ill-posed problems, whose matrix is a Kro-necker product. Problems of this kind arise, for instance, from the discretization of Fredholm integral equations of the first kind in two space-dimensions with a separable kernel. The available data (right-hand side) of many linear discrete ill-posed problems that arise in applications is contaminated by measurement errors. Straight-forward solution of these problems generally is not meaningful because of severe error propagation. We discuss how to combine the truncated singular value decomposition (TSVD) with reduced rank vector extrapolation to determine computed approximate solutions that are fairly insensitive to the error in the data. Exploitation of the structure of the problem keeps the computational effort quite small. © 2010 Elsevier Inc. All rights reserved.
1. Introduction
We consider linear discrete ill-posed problems of the form
A1X AT2
=
B, (1.1)where at least one of the matricesA1, A2
∈
R
n×n is of ill-determined rank. The singular values of matrices of ill-determined rank “cluster” at the origin. This makes the solutionX∈
R
n×nof (1.1) very sensitive to perturbations in the right-hand sideB∈
R
n×n. The latter generally represents observations that are contaminated by measurement errors of unknown size, i.e.,∗Corresponding author.
E-mail addresses: [email protected] (A. Bouhamidi), [email protected] (K. Jbilou), reichel@ math.kent.edu(L. Reichel),[email protected](H. Sadok).
0024-3795/$ - see front matter © 2010 Elsevier Inc. All rights reserved. doi:10.1016/j.laa.2010.06.001
B
=
B+
E, (1.2) whereBdenotes the unavailable error-free right-hand side and the matrixE∈
R
n×nrepresents the error. The norm ofEis not assumed to be known.Discrete ill-posed problems of the form (1.1) arise, for instance, from the discretization of Fredholm integral equations of the first kind in two space-dimensions,
Ω K
(
x, y, s, t)
f(
s, t)
dsdt=
g(
x, y)
,(
x, y)
∈
Ω, (1.3)
whereΩandΩare rectangles in
R
2and the kernel is separable, K(
x, y, s, t)
=
k1(
x, s)
k2(
y, t)
,(
x, y)
∈
Ω,(
s, t)
∈
Ω.
Discretization of (1.3) gives a matrix equation of the form (1.1).This paper describes a new structure-exploiting method for the solution of (1.1). The method uses properties of Kronecker products of two matrices. The Kronecker product,A
⊗
B, ofA∈
R
m×p and B∈
R
n×qis the(
mn)
×
(
pq)
matrixA⊗
B= [
aijB]
. The operatorvectransforms the matrixAinto the vectora∈
R
mpby stacking the columns. Assume that the sizes of the matricesA,B,C,D, andXare such that the matrix products in the following equations exist. Then(
A⊗
B)(
C⊗
D)
=
(
AC)
⊗
(
BD)
,(
A⊗
B)
−1=
A−1⊗
B−1, if A, Bare nonsingular, vec(
AXB)
=
(
BT⊗
A)
vec(
X)
,(
A⊗
B)
T=
AT⊗
BT;
(1.4)
see, e.g. [10, Chapter 4]. ForA, B
∈
R
m×p, we introduce the inner productA, BF=
tr(
ATB)
, where tr(
Z)
stands for the trace of the square matrixZ. The Frobenius norm, denoted by·
F, can be expressed asAF=
√
A, AF.In view of (1.4), the matrix equation (1.1) can be written as
Ax
=
b, A=
A2⊗
A1, x=
vec(
X)
, b=
vec(
B).
(1.5)The matricesA1andA2are assumed to be of moderate size, while their Kronecker product,A, is large. Our solution method for (1.1) first computes singular value decompositions of the matricesA1andA2, from which a truncated singular value decomposition (TSVD) ofAis determined. We describe how this can be done inexpensively without explicitly forming the large matrixAand all of its singular values and vectors. For a discussion on the application of the TSVD to linear discrete ill-posed problems; see, e.g. Hansen [8, Chapters 3 and 5]. Typically, only a few singular values and vectors ofAare required.
We apply a vector extrapolation method to the approximate solutions determined by the TSVD. This gives a sequence of approximate solutions
{
Xk}
k0, which approximate the desired solutionXof minimal Frobenius norm of the unavailable problemA1X AT2
=
B (1.6)with the error-free right-hand side associated with (1.1).
When the TSVD ofAis applied to the solution of (1.5) without extrapolation, the sequence of computed approximate solutions is very sensitive to the errore
=
vec(
E)
inb. The determination of a suitable truncation index for the TSVD ofAcan be a challenging task when no information about the norm ofeis available, even though a variety of techniques have been developed; see, e.g. [1,2,8,15]. A too large truncation index yields an approximate solution with an unnecessarily large contamination by propagated error, while a too small index gives an approximate solution with unnecessarily low resolution. Extrapolation generally furnishes a sequence of approximate solutions{
Xk}
k0that is not very sensitive to the errorEinB. This makes it easier to determine a suitable truncation index. This is illustrated with computed examples in Section4.The organization of this paper is as follows. In Section2we discuss the application of the TSVD to the solution of (1.1) and introduce notation. Section3shows how the Reduced Rank Extrapolation (RRE) method can be applied to the sequence of TSVD solutions, and Section4describes some numerical results. Concluding remarks can be found in Section5.
The extrapolation method of the present paper determines matrix sequences. An extrapolation method for computing vector sequences associated with approximate solutions of Fredholm integral equations of the first kind in one space-dimension previously has been discussed in [11]. The deriva-tion of the extrapolated sequence in this paper differs from [11] and so does the organization of the computations.
We have chosen the matricesA1 and A2 in (1.1) to be square and of the same size for ease of notation. These constraints easily can be removed. Moreover, it is straightforward to extend the method described to other related matrix equations, including analogues of (1.1) with tensor solutionsX. Matrix equations in control theory for which the solution technique of the present paper also may be of interest are discussed by Datta [3,4] and Datta and Datta [5].
2. The TSVD of a Kronecker product
Let the matricesA1andA2in (1.1) have the singular value decompositions A1
=
U1Σ1V1T, A2=
U2Σ2V2T,respectively. Here the matrices
Uk
= [
u1,k, u2,k,. . .
, un,k] ∈
R
n×n, Vk= [
v1,k, v2,k,. . .
, vn,k] ∈
R
n×n,have orthonormal columnsui,k
∈
R
n andvi,k∈
R
n, respectively, fori=
1,. . .
, nandk=
1,2. The diagonal entries ofΣk
=
diag[
σ
1,k,σ
2,k,. . .
,σ
n,k] ∈
R
n×nare the singular values ofAkfork
=
1,2. They are nonnegative and decreasing, i.e.,σ
1,kσ
2,k· · ·
σ
n,k0, k=
1,2;
see, e.g. [7, Chapter 2] for details on the singular value decomposition.
The singular value decomposition of the matrixA, defined in (1.5), is given by A
=
UΣVT.
The singular values ofA, i.e., the diagonal entries of
Σ
=
diag[
σ
1,σ
2,. . .
,σ
n2] ∈
R
n 2×n2are ordered so that
σ
1σ
2· · ·
σ
0> σ
0+1= · · · =
σ
n2=
0.
(2.1)They are an enumeration of the diagonal entries ofΣ2
⊗
Σ1, i.e.,σ
=
σ
j(),2σ
i(),1, 1n2, (2.2)
wherei
()
andj()
are nondecreasing functions ofwith range
{
1,2,. . .
, n}
. These functions are defined in Algorithm1of Section3. The columns of the orthogonal matricesU
= [
u1, u2,. . .
, un2] ∈
R
n 2×n2, V
= [
v1, v2,. . .
, vn2] ∈
R
n 2×n2,
are enumerations of the columns ofU2
⊗
U1andV2⊗
V1, respectively. Specifically, we have u=
uj(),2⊗
ui(),1, v=
vj(),2⊗
vi(),1, 1n2, (2.3) in agreement with (2.2).
For any 1k
0, the rank-kapproximationAkofAis defined by Ak
=
k =1
and its Moore–Penrose pseudoinverse can be expressed as A†k
=
k =1σ
−1 vuT.
Consider, for some 1k
0, the least-squares problem min
x∈Rn2
Akx−
b2.
(2.4)Here and throughout this paper
·
2denotes the Euclidean vector norm or the associated induced matrix norm. The minimal-norm solution of (2.4) is given byxk
=
A † kb=
k =1 uTbσ
v.
(2.5)Taking (2.2) and (2.3) into account, we can express (2.5) as xk
=
k =1 uj(),2⊗
ui(),1 T bσ
j(),2σ
i(),1 vj(),2⊗
vi(),1,=
k =1 uTj(),2⊗
uTi(),1 bσ
j(),2σ
i(),1 vj(),2⊗
vi(),1.
Application of the relations (1.4) yieldsxk
=
k =1 uTi(),1B uj(),2σ
i(),1σ
j(),2 vecvi(),1vTj(),2 ,whereb
=
vec(
B)
. Let the matrixXkbe defined byxk=
vec(
Xk)
. Then Xk=
k =1 uTi(),1B uj(),2σ
i(),1σ
j(),2 vi(),1vTj(),2.
Thus,Xkis a TSVD solution of (1.1). Its rank equalsk. In order forXkto be an accurate approximation of the minimal-norm solutionXof the error-free problem (1.6) associated with (1.1), it is important to choose a suitable value of the truncation indexk. This task is much simplified by extrapolating the sequence
{
Xk}
k0before selecting an index.3. RRE applied to TSVD sequences
The convergence of iterates determined by a slowly convergent iterative process often can be accel-erated by extrapolation methods. One of the most popular vector extrapolation methods is Reduced Rank Extrapolation (RRE) by Eddy [6] and Mesina [14]; see, e.g. [12,13] for more details on vector extrapolation methods.
Let
{
sp}
p0be a sequence of vectors inR
n 2and define the first and the second forward differences
sp
=
sp+1−
sp and 2sp=
sp+1−
sp.
When applied to the sequence
{
sp}
p0, the RRE method produces approximationst( q)p of the limit or anti-limit of thespasp
→ ∞
of the formtq(p)
=
q j=0γ
(q) j sp+j, whereq j=0
γ
(q) j=
1, q j=0η
ijγ
j(q)=
0, 0i<
q,with
η
ij=
(
2sp+i,sp+j)
. Here(
·
,·
)
denotes the standard inner product inR
n 2. Introduce the matrices
i
Sq,p
= [
isp,isp+1,. . .
,isp+q−1]
, i=
1,2.
Using Schur’s formula, the approximationtq(p)can be expressed as tq(p)
=
sp−
Sq,p 2 Sq,pT 2 Sq,p −1 2 STq,psp,wheres(qp)exists and is unique if and only if det
(
2Sq,pT 2Sq,p) /
=
0.Introduce the new approximation of the limit or anti-limit of the sequence
{
sp}
p0,˜
tq(p)=
q j=0γ
(q) j sp+j+1.
Following [13], we define the generalized residual
˜
r
tq(p)
= ˜
tq(p)−
tq(p), which can be written as˜
rtq(p)=
sp−
2Sq,p 2 ST q,p 2 Sq,p −1 2 ST q,psp.
We remark that when the sequence
{
sp}
p0is generated linearly, i.e., when sp+1=
(
I−
A)
sp+
b, p=
0,1,. . .
,the generalized residuals reduce to the standard residuals,
˜
rtq(p)
=
rtq(p)=
b−
A t(qp).
Henceforth, we focus on the case whenpis kept fixed and setp
=
0. For notational convenience, we denote the matricesiSq,0byiSqfori=
1,2, and the vectort(0)
q bytq. Then, fork1, we have tk
=
s0−
Sk−1 2 ST k−1 2 Sk−1 −1 2 ST k−1s0, (3.1) whereSk−1= [
s0,s1,. . .
,sk−1]
and2Sk−1= [
2s0,2s1,. . .
,2sk−1]
.We now apply the RRE method withp
=
0 to the sequence generated by (2.5) fork=
0,1,2,. . .
and obtain s0=
0, sk=
k =1δ
v, (3.2) whereδ
=
uTi(),1B uj(),2σ
i(),1σ
j(),2 , v=
vecvi(),1vTj(),2and the indicesi
()
andj()
are determined by the relations (2.1) and (2.2). We may assume that allδ
are nonvanishing, because if this is not the case, then we delete the corresponding member from the sequence (3.2) and compute the next one keeping the same index notation.It follows from (3.2) that the matrix TSVD sequence
{
Sk}
k0, such thatsk=
vec(
Sk)
, is given by Sk=
k l=1 uTi(),1B uj(),2σ
i(),1σ
j(),2 vi(),1vTj(),2.
Since
sk−1
=
sk−
sk−1=
δ
kvk, (3.3)the matrixSk−1
= [
s0,. . .
,sk−1]
can be factored according to Sk−1=
[δ
1v1,. . .
,δ
kvk]=
Vkdiag[
δ
1,. . .
,δ
k]
,whereVk
= [
v1,. . .
, vk]
. Moreover, since2sk−1=
δ
k+1vk+1−
δ
kvk,we deduce that2 Sk−1
=
Vk+1 ⎡ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎣−
δ
1δ
2−
δ
2 . .. . ..δ
k−
δ
kδ
k+1 ⎤ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎦.
(3.4) Therefore 2 SkT−1 2 Sk−1=
⎡ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎣δ
2 1+
δ
22−
δ
22−
δ
2 2δ
22+
δ
32−
δ
23 . .. . .. . ..−
δ
2 k−1δ
k2−1+
δ
k2−
δ
k2−
δ
2 kδ
2 k+
δ
2 k+1 ⎤ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎦.
Then, using (3.3) and (3.4), we obtain
2 ST k−1s0
= −
δ
2 1[
1,0,. . .
,0]
T.
It follows that the solution of the linear system of equations
2 ST k−1 2 Sk−1
α
(k)= −
2STk−1s0=
δ
2 1[
1,0,. . .
,0]
T is given byα
(k) i=
k j=i+1 1 δ2 j+1 k l=0 1 δ2 l+1 , 0i<
k.
Finally, the extrapolated vectortkdefined by (3.1) can be determined from tk
=
k =1α
(k) −1 uTi(),1B uj(),2σ
i(),1σ
j(),2 v, (3.5)where the indicesi
()
andj()
are given by the relations (2.1) and (2.2). LetTkbe defined bytk=
vec(
Tk)
. Then (3.5) yieldsTk
=
k =1α
(k) −1 uTi(),1B uj(),2σ
i(),1σ
j(),2 vi(),1vTj(),2.
(3.6)The expression
tk+1−
tkcan be helpful to determine when to terminate the computations. We obtain from (3.5) thattk+1
−
tk=
k j=1α
(k+1) j−1−
α
( k) j−1δ
j vj+
α
(k+1) k√
δ
k+1 vk+1.
Since the vectorsvj, 1jk+
1, are orthonormal, it follows that Tk+1−
TkF=
k j=1α
(k+1) j−1−
α
( k) j−1 2δ
j+
α
(k+1) k 2δ
k+1 ,whereTkis given by (3.6). For instance, one may acceptTk+1as an approximation ofXwhenTk+1is close toTk.
In the computed examples of Section4, we use the norm of the generalized residualRk, where
˜
rk
=
vec(
Rk)
, to determine a suitable truncation index. This norm easily can be evaluated using RkF=
1 k j=0 1 δ2 j+1.
We terminate the computations as soon as the relative generalized residual error satisfies
ρ
k= |
Rk+1F−
RkF|
/
RkFtol, (3.7)for a specified valuetol
>
0. The Reduced Rank Extrapolated-TSVD algorithm is summarized as follows:Algorithm 1. RRE-TSVD
•
Compute the SVD of the matricesA1andA2:[
Uk,Σk, Vk] =
svd(
Ak)
fork=
1,2.
•
Let:ui,k
=
Uk(
:
, i)
andvi,k=
Vk(
:
, i)
fori=
1,. . .
, nandk=
1,2. sk=
vec(Σk)fork=
1,2;z
= [
z1,. . .
, zn2]
T=
s2⊗
s1; w=
z;=
1;sum=
0; kopt=
1;•
fork=
1,2,. . .
1.Find the setIkof all the indicesi,such thatzi
=
max{
w}
; 2. fori∈
Ik(Use the indices i in increasing order)·
s=
zi;·
ii=
imodn;·
ifii=
0thenii=
n;end·
ji=
(
i−
ii)/
n+
1;·
i=
ii;j=
ji;·
=
+
1; end3.Setw
=
zk,where zk is the vector obtained from z by removing all the components of z with the indexi∈
Ik. 4.Compute·
δ
k=
ui,1BuTj ,2/
sk;·
sum=
sum+
1/δ
k2;·
γ
1=
1/
sum·
δ
21 ;·
α
1=
1−
γ
1; 5. Ifk>
1◦
SetS=
δ
1v1,1vT1,2(matrix of sizen×
n);T=
α
1S;◦
fori=
2,. . .
, k·
γ
i=
1/
sum·
δ
i2 ;·
α
i=
α
i−1−
γ
i;·
S=
S+
δ
ivii,1vjTi,2;·
T=
T+
α
iδ
ivii,1v T ji,2;◦
end◦
ifρ
ktol(cf.(3.7))·
kopt=
k−
1;·
stop◦
end end•
endThe functionsi
()
andj()
are defined implicitly in the beginning of thek-loop of the algorithm. Note that only a few singular values are extracted at a time. Since the algorithm generally terminates for a fairly small value ofk, this implies that only a few of the singular values ofA2⊗
A1are explicitly computed.4. Numerical examples
All computations are carried out using MATLAB 7.4 with machine epsilon about 2
×
10−16. We use the Regularization Tools package by Hansen [9] to define matricesA1andA2of ill-determined rank. The desired solution in Examples 4.1 and 4.2 are matricesXwith all entries unity. The error-free right-hand side is given byB=
A1X AT2and the associated error-contaminated right-hand sideBis determined by (1.2), where the error-matrixEhas normally distributed entries with zero mean and is normalized to correspond to a specific noise-levelν
=
EF BF.
(4.1) NeitherBnor
ν
are assumed to be known. All examples may be regarded at discretizations of Fredholm integral equations of the first kind in two space-dimensions. This is explicitly discussed in Example 4.3.Example 4.1.We let the nonsymmetric matricesA1, A2
∈
R
1500×1500be defined by the MATLAB func-tionsbaartandfoxgoodfrom [9]. Specifically, we letA1=
baart(
1500)
andA2=
foxgood(
1500)
. The computed condition numbers of these matrices areκ(
A1)
=
2×
1018andκ(
A2)
=
3×
1013, whereκ(
Ai)
=
Ai2A−i 12. Sinceκ(
A)
=
κ(
A1)κ(
A2)
, the matrixAis numerically singular. The noise-level in the right-hand sideBisν
=
1.
2×
10−2; cf. (4.1).Fig.4.1shows the behavior of approximate solutions determined by TSVD and RRE-TSVD. Note that whenkincreases and is sufficiently small, the relative error norms
Sk−
XF/
XF and Tk−
X/
XFdecrease; here theSkare determined by TSVD and theTkby RRE-TSVD. However, whenkis larger thankopt
=
6, the error in theSk increases rapidly withkwhile the norm of the error inTk stag-nates.We remark that since the error in the approximate solutionsTk determined by RRE-TSVD does not grow rapidly for increasing values ofk, it is much easier to determine an accurate approximation ofXfrom the extrapolated sequence
{
Tk}
k>0 than from the sequence{
Sk}
k>0; it suffices to choose k6.Example 4.2.For this experiment, we generate the nonsymmetric matricesA1
∈
R
2000×2000andA2∈
R
2000×2000with the codesbaartandursell, respectively, from [9]. The condition numbers of the two matrices are
κ(
A1)
=
7×
1018andκ(
A2)
=
2×
1013. Thus, the matrixA=
A2⊗
A1is numerically singular. The right-hand sideBis generated similarly as in Example 4.1 with noise-levelν
=
1.
3×
10−2.In order to determine whether the generalized residualsRkhave stagnated, we compute the quo-tients (3.7) for increasing values ofk. Letk
ˆ
denote the smallest index, such that|
Rˆk+1F−
RˆkF|
RkˆFtol
=
1×
10−4.
(4.2)We say that stagnation occurs atk
= ˆ
kand useTˆkas an approximation ofX. The choice the value of tolin (4.2) is based on numerical experience; we have found it to give good results for many discrete ill-posed problems.Fig. 4.1.Example 4.1: Relative errorSk−XF/XFfor TSVD (dashed graph) and relative errorTk−XF/XFfor RRE-TSVD
(solid graph).
Table4.1 shows the best truncation index to bekopt
=
8. The TSVD solutionS8 is an accurate approximation ofX, butS9is not. The error in the approximate solutionT8determined by RRE-TSVD is slightly larger than inS8, however, the error inT9 is the same as inT8. The stopping rule (4.2) determines the approximate solutionT9. We conclude that the best TSVD solution may be a somewhat better approximation of the desired solution than the best RRE-TSVD solution, but the latter method comes with a useful stopping criterion.Example 4.3.In this example, we consider the Fredholm integral equation
Ω K
(
x, y, s, t)
f(
s, t)
dsdt=
g(
x, y)
,(
x, y)
∈
Ω, (4.3)
whereΩ
= [
0,π/
2] × [
0,π/
2]
andΩ= [
0,π
] × [
0,π
]
. Let the kernel be given by K(
x, y, s, t)
=
k1(
x, s)
k2(
y, t)
,(
x, y)
∈
Ω,(
s, t)
∈
Ω,and define
g
(
x, y)
=
g1(
x)
g2(
y)
,Table 4.1
Example 4.2: A1=baart(2000), A2=ursell(2000), and noise-level ν=1.3×10−2.
k Tk−XF/XF Sk−XF/XF |Rk+1F− Rk|F/RkF
8 2.51×10−2 2.38×10−2 1.37×10−2 9 2.51×10−2 1.32×10−1 3.08×10−5
Fig. 4.2.Example 4.3: Relative errorSk−XF/XFfor TSVD (dashed graph) and relative errorTk−XF/XFfor RRE-TSVD
(solid graph).
Fig. 4.3.Example 4.3: ApproximationT23of Xdetermined by the RRE-TSVD method. No spurious frequency high-amplitude oscillations can be seen despite the large index number.
Fig. 4.4.Example 4.3: ApproximationS23ofX determined by the TSVD method. Spurious oscillations makeS23a poor approximant ofX.
where
ki
(
s, x)
=
exp(
scos(
x))
, gi(
s)
=
2 sinh(
s)/
s, i=
1,2.
We use the MATLAB code baart from [9] to discretize (4.3) and obtain two matrices A1, A2
∈
R
2500×2500and a scaled approximationX∈
R
2500×2500of the exact solutionf(
t, s)
=
sin(
t)
sin(
s)
. The error-free right-hand side of (1.1) is determined byB=
A1X AT2. Adding an error with noise-levelν
=
1×
10−2, we obtain the right-handBof (1.1). The matrixA=
A2⊗
A1is numerically singular. Fig.4.2shows that the norm of the error in the TSVD solutions decreases until iterationkopt=
10 and then grows rapidly withkfork14. The norm of the error of RRE-TSVD solutions can be seen to stagnate fork12. The stopping criterion (4.2) terminates the computations atkˆ
=
13, i.e.,T13is accepted as an approximation ofX.To illustrate the stability of the RRE-TSVD method, we display in Fig.4.3the last approximation, T23, generated for Fig.4.2. The plot of Fig.4.3shows no spurious high-frequency high-amplitude oscil-lations caused by propagated error. For comparison, Fig.4.4displays the corresponding approximate solution,S23, determined by TSVD. The spurious oscillations caused by propagated error makeS23a poor approximant ofX.
5. Conclusion
This paper describes a new extrapolated TSVD method for separable linear matrix ill-problems problems. The numerical examples illustrate that the choice of truncation index is not very critical, in the sense that the error in the computed approximations of the desired solution is not amplified as the truncation index is increased. Therefore, the extrapolated TSVD method gives useful approximate solution for all truncation indices sufficiently large. This is different from the situation for approximate solutions determined by the (unextrapolated) TSVD method. When the truncation index in the latter
method is chosen too large, the computed approximation is useless due to severe error propagation. The numerical examples illustrate that the truncation index for the extrapolated TSVD method can be chosen by monitoring the relative generalized residual or the differences
Tk+1−
Tk. A small relative generalized residual or stagnation of the mappingk→
Tk+1−
Tktypically indicates thatTk+1is a useful approximation of the desired solutionXof the unavailable error-free problem (1.6).References
[1] C. Brezinski, G. Rodriguez, S. Seatzu, Error estimates for linear systems with applications to regularization, Numer. Algorithms 49 (2008) 85–104.
[2] C. Brezinski, G. Rodriguez, S. Seatzu, Error estimates for the regularization of least squares problems, Numer. Algorithms 51 (2009) 61–76.
[3] B.N. Datta, Linear and numerical linear algebra in control theory: some research problems, Linear Algebra Appl. 197–198 (1994) 755–790.
[4] B.N. Datta, Stability and inertia, Linear Algebra Appl. 302–303 (1999) 563–600.
[5] B.N. Datta, K. Datta, The matrix equationXA=ATXand an associated algorithm for solving the inertia and stability problems, Linear Algebra Appl. 97 (1987) 103–119.
[6] R.P. Eddy, Extrapolation to the limit of a vector sequence, in: P.C.C. Wang (Ed.), Information Linkage Between Applied Mathematics and Industry, Academic Press, New York, 1979, pp. 387–396.
[7] G.H. Golub, C.F. Van Loan, Matrix Computations, third ed., Johns Hopkins University Press, Baltimore, 1996. [8] P.C. Hansen, Rank-Deficient and Discrete Ill-Posed Problems, SIAM, Philadelphia, 1998.
[9] P.C. Hansen, Regularization tools version 4.0 for MATLAB 7.3, Numer. Algorithms 46 (2007) 189–194. [10] R.A. Horn, C.R. Johnson, Topics in Matrix Analysis, Cambridge University Press, Cambridge, England, 1991.
[11] K. Jbilou, L. Reichel, H. Sadok, Vector extrapolation enhanced TSVD for linear discrete ill-posed problems,Numer. Algorithms 51 (2009) 195–208.
[12] K. Jbilou, H. Sadok, Analysis of some vector extrapolation methods for linear systems, Numer. Math. 70 (1995) 73–89. [13] K. Jbilou, H. Sadok, Vector extrapolation methods, applications and numerical comparison, J. Comput. Appl. Math. 122
(2000) 149–165.
[14] M. Mesina, Convergence acceleration for the iterative solution ofx=Ax+f, Comput. Methods Appl. Mech. Engrg. 10 (1977) 165–173.
