2.7.1 Factor Matrix Operations
Certain transformations on the CP/Tucker factor matrices have a useful effect on the output tensor eT that results from Eqs. 2.1 to 2.5. Factor rows map to axis- aligned hyperslices from the input, while factor columns (ranks) map to axis- aligned hyperslices of the transform. Removing rows prior to reconstruction re- sults in a subtensor of the original input, while removing ranks results in a coarser approximation. Furthermore, if instead of a matrix U(n)we use a linear combi- nation of its rows u(n) := cTU(n) (where c is a vector withI
n entries), then the
reconstruction produces the combination of the corresponding hyperslices along then-th mode. In the Tucker case, this means thatB ×1U(1)×2...×nu(n)×n+1
...×N U(N ) equals
In
X
k=1
c[k]· eT [..., k, ...] = eT ×ncT, (2.8)
where we have written a tensor-times-vector (TTV) product on the right-hand side (a mode-n TTV computes the dot product between a vector and each mode-n ten- sor fiber). For matrices, it holds thatB ×1 U(1)×2 ...×n(A· U(n))×n+1...×N
U(N ) = eT ×nA. In particular, we can downsample the factors column-wise to
obtain downsampled versions of our data. This principle was first exploited in vol- ume visualization by [Suter et al., 2013] and further analyzed in [Ballester-Ripoll et al., 2015], [Ballester-Ripoll and Pajarola, 2015]. We frequently use similar ma- nipulations in this thesis, especially in Ch. 4 and Ch. 5. Last, note that equivalent spatial manipulation properties hold for many other tensor decompositions; for example in TT one must use combinations of core slices instead of matrix rows.
2.7.2 TT Format
Multiplication/division of a TT tensor by a scalar α is achieved by simply mul- tiplying/dividing one of its cores (say, the first) by α [Oseledets, 2011]. Tensor- tensor addition is written as(T1+T2)[x] :=T1[x] +T2[x] and has the following
2.7 Operating with Tensors 19 T1(1)[x1] T2(1)[x1] (first core) T1(n)[xn] 0 0 T2(n)[xn] ! (1 < n < N ) T1(N )[xN] T2(N )[xN] ! (last core)
To subtract two tensors, we use the same addition method but flip first the second tensor’s sign (done by flipping the sign of, say, its first core).
The element-wise (or Hadamard) product(T1 ◦ T2)[x] := T1[x]· T2[x] arises
from a slice-wise Kronecker product:
(T1(1)[x1]⊗ T2(1)[x1])· ... · (T1(N )[xN]⊗ T2(N )[xN]) (2.9)
This product has many applications. Convolution between two N-dimensional TT tensors, for example, can be obtained by computing each tensor’s Fourier transform (FT) along the spatial dimension on every TT core, followed by element- wise product and inverse FT [Rakhuba and Oseledets, 2015].
2.7.3 Adaptive Cross-Approximation
The so-called adaptive cross-approximation (ACA) for the TT format [Oseledets and Tyrtyshnikov, 2010b], [Savostyanov and Oseledets, 2011] is a type of pro- gressive sampling scheme that allows to approximate all mentioned operations and others at cost O(IN R3) at most, i.e. devoid of the curse of dimensionality.
These include arbitrary element-wise functions, differentiation, integration, con- volution, and more [Cichocki et al., 2016], [Lee and Cichocki, 2017]. The ranks needed may grow as a result of such operations. It is crucial to keep them rea- sonably low at all stages of any computational pipeline, otherwise the benefits of tensor compression vanish. An error-bounded rounding algorithm called TT- round [Oseledets, 2011] exists to re-compress down any tensor when needed by means of a sequence ofN− 1 SVD truncations. ACA is exploited and covered in more detail in Ch. 6; see also App. A.
Global Optimization
ACA has been successfully used to find the (approximately) maximal element in modulus of a tensor [Dolgov et al., 2014], [Oferkin et al., 2015], as it was empirically found that the subtensors accessed during ACA very often contain such maximal elements. The variant known as rectangular maxvol is a tool even
more efficient for this task [Mikhalev and Oseledets, 2015] and is the one we use (released in [ttp, ]). This effectively allows solving global optimization problems in the TT format, a very attractive feature we make use of in Ch. 6 and 7.
3
C
H
A
P
T
E
R
TENSOR COMPRESSION
3.1
Overview
As outlined earlier, this part of the dissertation is devoted to dense tensors. The challenge we tackle first is multidimensional compression, which is a significant actor in data-intensive visualization. Two frequent goals in scientific or visual computing applications are a) to reduce the complex initial input to alleviate com- putational bottlenecks, while b) aiming for a faithful and efficient reconstruction. This is the case when memory or time restrictions are an issue. Lossy compression is often the prescribed strategy, since complex and large data sets rarely benefit much from lossless compression (especially if they use floating-point precision). If the compressed data set is to be used for subsequent computational analysis and/or to be fed into simulation routines, typically very small errors are tolerated. Conversely, if visualization is to follow decompression, then higher error rates are acceptable.
Depending on the specific application, sometimes certain additional targets are desirable. These include fast support for random-access decompression, fine com- pression rate granularity, asymmetry (faster decompression than compression), bounded error, support for higher-dimensions, ease of parallelization, etc. For these reasons the compression problem is both broad and challenging, and no catch-all solution exists as of yet.
Contribution
We introduceTTHRESH, a tensor compressor for visualization applications whose foremost priority is data reduction at high compression ratios. In particular, while the ratios we achieve at low error tolerance are reasonable, we outperform state- of-the-art methods on the higher error spectrum. Our algorithm also possesses advantages that are inherent to multilinear transforms in general and tensor de- compositions in particular, including support for linear manipulation of the data set in the compressed domain (recall Sec. 2.7.1).
Definitions
The RMSE (root-mean-square error) between a tensor of size I1 × · · · × IN and
an approximation eT is kT − eT k/√I1· · · IN. For our experiments we use the
normalized PSNR (peak signal-to-noise ratio) in terms of the RMSE as follows:
PSNR(T , eT ) = 20 · log10
max{T } − min{T } 2· RMSE(T , eT )
!