Low-Rank Tensor Formats

µ,ν∈∇^d (5.8)

defines an elliptic operator on `₂(∇^d). We additionally assume the Ψ_ν to be L₂-orthonormal, and hence the corresponding eigenvalue problem reads

X

µ

sνhΨν, HΨµisµ
uµ= λs²_νuν, ν ∈ ∇^d, (5.9)

where u = (uν)_ν∈∇^d = (s⁻¹_ν hu, Ψνi)ν∈∇^d for u∈ H¹(R^d), or briefly, Hu = λs²u. Note that by the Riesz basis property,kukH¹(R^d) ∼ kuk`2(∇^d).

5.2.1 Low-Rank Tensor Formats

Let us first consider low-rank approximation in the case of two-dimensional problems. Let u ∈

`₂(Λ₁× Λ2) with finite Λ₁, Λ₂ ⊂ ∇, then singular value decomposition yields a representation in the form

u=

r

X

k=1

(U⁽¹⁾_k ⊗ U⁽²⁾_k ) σ_k, (5.10)

where σ1 ≥ · · · ≥ σr > 0 and, for i ∈ {1, 2} and k, l ∈ {1, . . . , r}, U⁽ⁱ⁾_k = (U_k,ν⁽ⁱ⁾)ν∈Λi with supp U⁽ⁱ⁾_k ∈ Λi and hU⁽ⁱ⁾_k , U⁽ⁱ⁾_l i = δkl.

Here r is the rank of the matrix (u_ν1,ν2)_{ν1∈Λ1,ν2∈Λ2}, and the decomposition (5.10) also provides a means of computing the best approximation by an element of `₂(Λ₁× Λ2) with rank ˜r < r: by the Eckart-Young theorem [47], the error

u−

˜ r

X

k=1

(U⁽¹⁾_k ⊗ U⁽²⁾_k )σ_k =

 X^r

k=˜r+1

|σk|²¹₂

(5.11)

is minimal among all rank-˜r approximations of u.

This can be extended to u∈ `2(∇²) without the restriction of finite support, in which case the singular value decomposition is replaced by the Hilbert-Schmidt decomposition of operators. For such a general u, the infinite matrix (u_ν1,ν2)_{ν1∈∇,ν2}∈∇ defines a Hilbert-Schmidt operator

Tu: `<sub>2</sub>(∇) → `2(∇) , c 7→ X

ν∈∇

u_ν,ν˜ c_ν

˜ ν∈∇, and the spectral theorem yields a decomposition

u=

∞

X

k=1

(U⁽¹⁾_k ⊗ U⁽²⁾_k ) σ_k, (5.12)

with a nonnegative nonincreasing sequence (σ_k)_k∈N∈ `2(N), and orthonormal bases{U⁽ⁱ⁾_k }k∈N of

`₂(∇) for i ∈ {1, 2}. The low-rank approximation property (5.11) carries over to this case, where in general r =∞.

At first glance, a natural extension of (5.10) to the higher-dimensional case would be a repre-sentation of u∈ `2(∇^d) of the form

u=

r

X

k=1

(U⁽¹⁾_k ⊗ · · · ⊗ U^(d)_k ) a_k (5.13)

with kU⁽ⁱ⁾_k k = 1 and ak ∈ R. This is typically referred to as canonical format, canonical polyadic

5.2 Tensor Structures for Wavelet Coordinates

decomposition, or parallel factors. The smallest r∈ N0∪{∞} for which such a representation exists is referred to as the canonical rank of u. For our purposes, the major problem with this type of representation is the lack of a sufficiently reliable recompression procedure. In fact, the problem of approximating a given u by an expansion (5.13) of specified rank is in general ill-posed [39], which additionally necessitates a suitable regularization. For such regularized problems, however, one still needs to rely on minimization procedures that generally cannot be guaranteed to converge to the global minimum.

In this regard, a representation in the form

u= has substantially more favorable properties, and in this chapter we will be dealing with this format, and remark on generalizations with similar features. Here the order-d tensor a is referred to as core tensor, the matrix U⁽ⁱ⁾ with column vectors U⁽ⁱ⁾_k ∈ `2(∇), k = 1, . . . , ri, as the i-th mode frame.

This is the so-called Tucker format [144, 145] or subspace representation. Note that for ˆd∈ N, we can also represent u∈ `2(∇<sup>ddˆ</sup> ) in the form (5.14) with U<sup>(i)</sup><sub>k</sub> ∈ `2(∇^dˆ). For the sake of simplicity, we consider the case ˆd = 1 in what follows, but one can proceed completely analogously for general ˆd.

Clearly, any compactly supported u ∈ `2(∇<sup>d</sup>) can be represented in the form (5.14) for some r ∈ N<sup>d</sup>0. For general u ∈ `2(∇^d), the sum in (5.14) may be infinite. We correspondingly define rank(u)∈ (N0∪ {∞})^d by

rank(u)_i := dim span{U⁽ⁱ⁾_k : k∈ N} , i = 1, . . . , d . (5.15) This vector is referred to as the multilinear rank of u. Note that in a representation of the form (5.14), one can always orthogonalize the columns of U⁽ⁱ⁾ to obtainhU⁽ⁱ⁾_k , U⁽ⁱ⁾_l i = δkl for all i. We shall refer to U⁽ⁱ⁾ with the latter property as orthonormal mode frames.

Remark 5.1. As we have seen in Section 4.3, when approximating the wavelet coefficients of hydrogenic ground states in the format (5.14), one can achieve exponential decrease of the H¹ -approximation error with respect to the multilinear ranks.

In the case of helium or hookium, fast convergence of the representation with respect to the ranks can only be expected when – assuming single-electron coordinates x, y∈ R³ – the coordinate pairs(x_i, y_i) for i = 1, 2, 3 are not separated. We have shown for hookium in Section 4.3 that the corresponding wavelet coefficients u ∈ `2((∇<sup>2</sup>)<sup>3</sup>) can be represented efficiently in the form (5.14) withd = 3 and U<sup>(i)</sup><sub>k</sub> ∈ `2(∇²), where each U⁽ⁱ⁾ corresponds to a coordinate pair (xi, yi).

To simplify notation for the sums in (5.14), for r∈ N^d0 we define

Kd(r) := for the corresponding vectors with entry i deleted. We shall also need the auxiliary quantities

a⁽ⁱ⁾_pq :=

5 Adaptive Wavelet Schemes for Tensor Representations

There exists an analogue of the singular value decomposition of matrices, the higher-order sin-gular value decomposition [107], for the Tucker tensor format (5.14). In the following theorem, we summarize its properties in the more general case of sequence spaces, where the the singular value decomposition is replaced by the spectral theorem for compact operators.

Theorem 5.2. For any u∈ `2(∇^d) there exist orthonormal mode frames{U⁽ⁱ⁾_k }k∈N, i = 1, . . . , d,

Proof. The following is essentially an adaptation of the arguments for the finite-dimensional case given in [107] to the infinite-dimensional sequence space `₂(∇^d).

Let u = (u_ν)_ν∈∇d ∈ `2(∇^d). For each i∈ {1, . . . , d} we consider the mode-i matricization of u,

By the spectral theorem, for each i there exist a nonnegative real sequence (σn⁽ⁱ⁾)_n∈N, where σ⁽ⁱ⁾n are the eigenvalues of (T⁽ⁱ⁾)^∗T⁽ⁱ⁾
1/2

The representation (5.19) converges in the Hilbert-Schmidt norm, and as a consequence we have u= X

5.2 Tensor Structures for Wavelet Coordinates

which by orthonormality of{U⁽ⁱ⁾n }n∈N yields

= σ⁽ⁱ⁾_p σ⁽ⁱ⁾_q X orthonormality of{Vn⁽ⁱ⁾}n∈N. Property (iii) follows with the observations

The additional properties of the decomposition for finitely supported u are clear, since in this case the spectral decomposition reduces to a finite-dimensional singular value decomposition.

Remark 5.3. The result of Theorem 5.2 holds analogously for u∈ `2(∇<sup>d ˆd</sup>) and U<sup>(i)</sup><sub>k</sub> ∈ `2(∇^dˆ) with d, ˆd∈ N. As noted in Remark 5.1, this is relevant for two-electron systems such as helium, where we use a decomposition withd = 3 and ˆd = 2.

Note that by analogy to (5.10), the σ_k⁽ⁱ⁾ are also referred to as mode-i singular values of u.

Property (iii) in Theorem 5.2 leads to a simple procedure for truncation to lower multilinear ranks with an explicit error estimate in terms of the mode-i singular values. In this manner, one does not necessarily obtain the best approximation for prescribed rank, but the approximation is quasi-optimal in the sense that the error is at most by a factor √

d larger than the error of best approximation with the same multilinear rank.

In principle, a representation as in Theorem 5.2 can be obtained for any u ∈ `2(∇^d) by a combination of standard linear algebra procedures. For our purposes, the relevant task is to obtain such a representation for finitely supported u given in the form

u= X

without further assumptions on ˜U⁽ⁱ⁾ and ˜a, where in particular the columns of the ˜U⁽ⁱ⁾ may be linearly dependent. From the arguments in [107], one can extract the well-known procedure described in Algorithm 5.1 that yields a representation as in Theorem 5.2 for finitely supported u.

Remark 5.4. Assuming supp u⊆ Λ1× · · · × Λd, hosvd as in Algorithm 5.1 can be performed in O(|˜r|^∞Q

i˜r_i+P

ir˜²_i#Λ_i) operations, using only standard linear algebra operations.

The procedure hosvd can be interpreted as the selection of basis functions, given by the columns of U⁽ⁱ⁾, which are adapted to a given u. However, the tensor a containing the coefficients for

5 Adaptive Wavelet Schemes for Tensor Representations

Algorithm 5.1 [{U⁽ⁱ⁾}, a] = hosvd({ ˜U⁽ⁱ⁾}, ˜a) input u in representation (5.21)

output mode frames U⁽ⁱ⁾ and core tensor a as in Theorem 5.2

1: for all i∈ {1, . . . , d}

2: perform QR–factorization ˜U⁽ⁱ⁾= Q⁽ⁱ⁾R⁽ⁱ⁾

. Q⁽ⁱ⁾ ∈ R^{Λ(i)×{1,...,ri}}, R⁽ⁱ⁾ ∈ R^{{1,...,ri}×{1,...,˜ri}}, where r≤ ˜r.

. columns of Q⁽ⁱ⁾ are orthonormal, R⁽ⁱ⁾ is right upper triangular

3: end for

4: ˆa_k←P

l∈Kd(˜r)R⁽¹⁾_k

1,l1· · · R_k^(d)_d,ld˜a_l for k∈ Kd(r)

5: for all i∈ {1, . . . , d}

6: build matricization T⁽ⁱ⁾ ∈ R^{{1,...,ri}×Ii} with entries T⁽ⁱ⁾

ki,ˇki := ˆa_k1,...,kd

. Ii ={1, . . . , r1} × · · · × {1, . . . , ri−1} × {1, . . . , ri+1} × . . . × {1, . . . , rd}

7: perform singular value decomposition T⁽ⁱ⁾ = V⁽ⁱ⁾Σ⁽ⁱ⁾(W⁽ⁱ⁾)^T

8: U⁽ⁱ⁾← Q⁽ⁱ⁾V⁽ⁱ⁾

9: end for

10: a_k←P

l∈Kd(r)V_l⁽¹⁾

1,k1· · · V_l^(d)_d,kdˆa_l for k ∈ Kd(r)

these basis functions has Q

ir_i components, which even for moderate multilinear ranks becomes too expensive for larger d.

A possible alternative for higher dimensions is the Hierarchical Tucker or H–Tucker format [75].

It has similar features as the Tucker format, and there exists a scheme that parallels hosvd in its properties [64], but has storage and work complexity linear in d when applied to input tensors given in the appropriateH–Tucker representation. This comes at the price of additional structural constraints on the tensors in the form of a modified notion of tensor ranks. For the model problems that we are considering in this chapter, theH–Tucker format does not give an advantage over the Tucker format, and hence what follows will be formulated for the latter. The schemes developed here can, however, be applied to the H–Tucker format without major modifications; we will come back to this point in Section 5.3.3.