On Efficiently Updating Singular Value Decomposition
Based Reduced Order Models
Ralf Zimmermann
1. Input:
CFD snaphots
Flow solutions at different flow conditions
The POD-based ROM approach
1. Input:
CFD snaphots
Flow solutions at different flow conditions2. POD Basis
Orthonomal basis ordered by infor- mation content spanning the same spaceThe POD-based ROM approach
1. Input:
CFD snaphots
Flow solutions at different flow conditions2. POD Basis
Orthonomal basis ordered by infor- mation content spanning the same spaceThe POD-based ROM approach
4. Prediction step
4. Prediction step
1. Input:
CFD snaphots
Flow solutions at different flow conditions2. POD Basis
Orthonomal basis ordered by infor- mation content spanning the same spaceThe POD-based ROM approach
5. Output:
approximated flow field
Untried flow condition
1. Input:
CFD snaphots
Flow solutions at different flow conditions2. POD Basis
Orthonomal basis ordered by infor- mation content spanning the same spaceThe POD-based ROM approach
4. Prediction step
4. Prediction step
POD based reduced order models are a powerful tool…
Industrial aircraft configuration
… but treating large snapshots may become a challenge!
POD/SVD representation known
Nomenclature
T m m m m m T m m m m j j m m m n m mV
U
Y
W
W
A
W
A
W
A
Y
Y
W
A
R
W
W
Y
:
SVD
),
,...,
(
:
)
,...,
(
1
:
Centering
:
average
Snapshot
)
,...,
(
:
matrix
Snapshot
1 1 1 1 11
,
)
(
:
content
n
Informatio
Relative
1 2 1 2
m
r
r
ric
m
mm
i i m r i i
Discard columns corresponding to small singular values:
Definition:
The data set
is called reduced-order model of order of
The ratio is called the compression rate.
m m m m m r m r m r m r n r m T m m m mR
V
V
V
R
U
U
U
V
U
Y
0
0
0
0
0
0
,
,...,
,
,...,
,
1 1 1
U
m
,
m
,
V
m
,
A
m
,
n
,
m
,
r
m
m
r
Y
m
.
r
mReduced-order representation
Given:
ROM of
p new snapshot observations
Task: Compute ROM
of
Requirement:
use only the previous stage ROM and the incoming snapshots!
! Keep computational costs depending on as low as possible
U
m
,
m
,
V
m
,
A
m
,
n
,
m
,
r
m
Y
m
R
n
m
.
.
p
m
Y
m
m
p
W
W
1
,...,
U
m
p
,
m
p
,
V
m
p
,
A
m
p
,
n
,
m
p
,
r
m
p
The SVD basis update problem
m
Objective
POD MODES
=
SVDSNAPSHOTS
Efficient SVD basis update
Objective
Efficient SVD basis update
•
Update SVD basis without having to store the
initial
snapshots
POD MODES
=
SVD SNAPSHOTS=
SNAPSHOTS SVD+
+
POD MODESObjective
POD MODES=
SVD SNAPSHOTS SNAPSHOTS+
SNAPSHOTS=
SVD POD MODESEfficient SVD basis update
Updating the snapshot mean
Shift vector:
Shift update snapshots to the previous-stage center:
To do:
Decompose
SVD basis update strategies
p
m
m
p
m
p
m
m
i
i
m
p
m
p
m
T
A
A
W
pA
T
1
1
:
p
i
A
W
W
m
i
:
m
i
m
,
1
,...,
,
1
,
!
T
T
V
U
T
W
Y
Y
Updating the snapshot mean
Shift vector:
Shift update snapshots to the previous-stage center:
To do:
Decompose
SVD basis update strategies
p
m
m
p
m
p
m
m
i
i
m
p
m
p
m
T
A
A
W
pA
T
1
1
:
p
i
A
W
W
m
i
:
m
i
m
,
1
,...,
,
1
,
!
T
T
V
U
T
W
Y
Y
start here
Setting
it holds
SVD basis update strategies (A): two-steps SVD
1,2
n
p
T
p
m
p
p
m
B
I
Y
X
,
0
,
0
,
T
r
p
m
m
m
T
m
mV
U
X
B
W
X
W
Y
,
and
,
0
Setting
it holds
SVD basis update strategies (A): two-steps SVD
1,2
n
p
T
p
m
p
p
m
B
I
Y
X
,
0
,
0
,
T
r
p
m
m
m
T
m
mV
U
X
B
W
X
W
Y
,
and
,
0
Setting
it holds
Factoring out orthogonal components:
SVD basis update strategies (A): two-steps SVD
1,2
n
p
T
p
m
p
p
m
B
I
Y
X
,
0
,
0
,
T
r
p
m
m
m
T
m
mV
U
X
B
W
X
W
Y
,
and
,
0
)
,
0
(
)
0
,
(
0
0
,
p
p
m
p
p
r
T
m
p
p
m
m
T
I
V
I
W
U
B
W
X
mSetting
it holds
Factoring out orthogonal components:
where
e.g. via Gram Schmidt
SVD basis update strategies (A): two-steps SVD
1,2
n
p
T
p
m
p
p
m
B
I
Y
X
,
0
,
0
,
T
r
p
m
m
m
T
m
mV
U
X
B
W
X
W
Y
,
and
,
0
)
,
0
(
)
0
,
(
0
,
p
p
m
p
p
r
T
m
T
orth
T
m
m
orth
m
T
I
V
P
P
W
U
P
U
B
W
X
m)
(
orth
),
(
U
W
P
P
U
W
P
m
T
orth
Setting
it holds
Factoring out orthogonal components:
SVD basis update strategies (A): two-steps SVD
1,2
n
p
T
p
m
p
p
m
B
I
Y
X
,
0
,
0
,
T
r
p
m
m
m
T
m
mV
U
X
B
W
X
W
Y
,
and
,
0
)
,
0
(
)
0
,
(
0
,
p
p
m
p
p
r
T
m
T
orth
T
m
m
orth
m
T
I
V
P
P
W
U
P
U
B
W
X
morthonormal
columns
Setting
it holds
Factoring out orthogonal components:
SVD basis update strategies (A): two-steps SVD
1,2
n
p
T
p
m
p
p
m
B
I
Y
X
,
0
,
0
,
T
r
p
m
m
m
T
m
mV
U
X
B
W
X
W
Y
,
and
,
0
)
,
0
(
)
0
,
(
0
,
p
p
m
p
p
r
T
m
T
orth
T
m
m
orth
m
T
I
V
P
P
W
U
P
U
B
W
X
morthonormal
columns
Setting
it holds
Factoring out orthogonal components:
SVD basis update strategies (A): two-steps SVD
1,2
n
p
T
p
m
p
p
m
B
I
Y
X
,
0
,
0
,
T
r
p
m
m
m
T
m
mV
U
X
B
W
X
W
Y
,
and
,
0
)
,
0
(
)
0
,
(
0
,
p
p
m
p
p
r
T
m
T
orth
T
m
m
orth
m
T
I
V
P
P
W
U
P
U
B
W
X
msize
(
r
m
p
)
(
r
m
p
)
Setting
it holds
Factoring out orthogonal components:
SVD basis update strategies (A): two-steps SVD
1,2
n
p
T
p
m
p
p
m
B
I
Y
X
,
0
,
0
,
T
r
p
m
m
m
T
m
mV
U
X
B
W
X
W
Y
,
and
,
0
)
,
0
(
)
0
,
(
~
~
~
,
p
p
m
p
p
r
T
m
T
orth
m
T
I
V
V
U
P
U
B
W
X
mSetting
it holds
Factoring out orthogonal components:
SVD basis update strategies (A): two-steps SVD
1,2
n
p
T
p
m
p
p
m
B
I
Y
X
,
0
,
0
,
T
r
p
m
m
m
T
m
mV
U
X
B
W
X
W
Y
,
and
,
0
)
,
0
(
)
0
,
(
~
~
~
,
p
p
m
p
p
r
T
m
T
orth
m
T
I
V
V
U
P
U
B
W
X
mRepeat for shifting to new center:
SVD basis update strategies (A): two-steps SVD
1,2
,
1
,
!
T
p
m
p
m
p
m
T
p
m
p
m
m
p
m
Y
W
T
U
V
Y
SVD now known
Comments:
‘
re-orthogonalization’ expensive, additional (large-scale) SVD required
less robust via Gram-Schmidt
Parallelization is more involved
SVD basis update strategies (A): two-steps SVD
1,2
)
(
orth
),
(
U
W
P
P
U
W
P
m
T
orth
Objective:
First step:
Compute SVD of
Reduce to symmetric EVD
SVD basis update strategies (B): EVD
3
+SVD
W
W
Y
W
W
Y
Y
Y
W
Y
W
Y
T
m
T
T
m
m
T
m
m
T
m
,
,
,
1
,
!
T
p
m
p
m
p
m
T
p
m
p
m
m
p
m
Y
W
T
U
V
Y
Y
m
,
W
Objective:
First step:
Compute SVD of
Reduce to symmetric EVD
SVD basis update strategies (B): EVD
3
+SVD
W
W
V
U
W
W
U
V
V
V
W
Y
W
Y
T
T
m
m
m
T
T
m
m
m
T
m
m
m
m
T
m
2
,
,
,
1
,
!
T
p
m
p
m
p
m
T
p
m
p
m
m
p
m
Y
W
T
U
V
Y
Y
m
,
W
Exploit previous
Objective:
First step:
Compute SVD of
Reduce to symmetric EVD
SVD basis update strategies (B): EVD
3
+SVD
p
p
T
m
T
m
m
T
T
m
m
m
p
p
m
m
T
m
I
V
W
W
U
W
W
U
I
V
W
Y
W
Y
0
0
0
0
,
,
2
,
1
,
!
T
p
m
p
m
p
m
T
p
m
p
m
m
p
m
Y
W
T
U
V
Y
Y
m
,
W
factor out
Objective:
First step:
Compute SVD of
Reduce to symmetric EVD
SVD basis update strategies (B): EVD
3
+SVD
p
p
T
m
T
m
m
T
T
m
m
m
p
p
m
m
T
m
I
V
W
W
U
W
W
U
I
V
W
Y
W
Y
0
0
0
0
,
,
2
,
1
,
!
T
p
m
p
m
p
m
T
p
m
p
m
m
p
m
Y
W
T
U
V
Y
Y
m
,
W
size
(
r
p
)
(
r
p
)
Objective:
First step:
Compute SVD of
Reduce to symmetric EVD
SVD basis update strategies (B): EVD
3
+SVD
p
p
T
m
T
p
p
m
m
T
m
I
V
Q
Q
I
V
W
Y
W
Y
0
0
~
~
~
0
0
,
,
,
1
,
!
T
p
m
p
m
p
m
T
p
m
p
m
m
p
m
Y
W
T
U
V
Y
Y
m
,
W
Objective:
First step:
Compute SVD of
Reduce to symmetric EVD
Compute left singular vectors via
SVD basis update strategies (B): EVD
3
+SVD
p
p
T
m
T
p
p
m
m
T
m
I
V
Q
Q
I
V
W
Y
W
Y
0
0
~
~
~
0
0
,
,
,
1
,
!
T
p
m
p
m
p
m
T
p
m
p
m
m
p
m
Y
W
T
U
V
Y
Y
m
,
W
~
~
1
,
ˆ
U
W
Q
U
Use Brand’s method for shifting to new center:
SVD basis update strategies (B): EVD
3
+ SVD
,
1
,
!
T
p
m
p
m
p
m
T
p
m
p
m
m
p
m
Y
W
T
U
V
Y
SVD now known
Objective:
Let
Reduce to symmetric EVD,
exploit previous stage SVD in matrix products
Compute left singular vectors via
SVD basis update strategies (C): one-step EVD
)
(
)
(
~
~
~
T
m
p
m
p
T
R
Q
Q
X
X
,
1
,
!
T
p
m
p
m
p
m
T
p
m
p
m
m
p
m
Y
W
T
U
V
Y
(
)
1
,
:
Y
m
W
T
m
p
T
m
p
R
n
m
p
X
1
~
~
X
Q
U
Comments:
Straight forward parallelization!
(only standard matrix products required in parallel)
Count flops for matrix product
Strategy (B) is more efficient than strategy (A):
Analysis of Computational costs
nmp
AB
,
A
R
n
m
,
B
R
m
p
)
)
((
))
(
(
)
(
flops(B)
flops(A)
2
2
2
1
n
p
mp
O
P
orth
O
n
p
p
p
r
m
The ranking of Strategy (B) vs. (C) depends on the compression rate!
Assumption:
Then the computational costs differ by
Analysis of Computational costs
(
2
)
(
2
3
)(
)
2
flops(B)
flops(C)
n
x
x
2m
2
x
mp
m
p
2
p
rate)
on
(compressi
,
r
m
m
p
m
mx
p
xm
p
r
r
The ranking of Strategy (B) vs. (C) depends on the compression rate!
Solving the quadratic equation shows that
Strategy (B) is more efficient than Strategy (C) if
In practical implementations, use switch to select the most efficient
update strategy.
Analysis of Computational costs
3
(
1
)
10
(
1
)
10
7
0
x
41m
p
m
p
m
2
p
2
p
mRules of thumb:
For highly compressed models, Strategy (B) is more efficient than
Strategy (C)
For weakly compressed models, the opposite holds true
More precisely
Strategy (B) is more efficient than Strategy (C), if
and
Strategy (C) is more efficient than Strategy (B), if
Analysis of Computational costs
2 1
x
.
3 2
x
.
1
2
p
m
Uncompressed models
Example: Updating SVDs of Random matrices
100
,
500
,
000
,
100
,
,
p
m
n
R
W
R
Y
n
m
n
p
Method
Reconstruction error
time (sec)
Strategy (A), SVD-orth
2.504e-12
7.88
Strategy (A), QR-orth
2.333e-12
8.15
Strategy (B)
2.342e-12
6.32
Strategy (C)
2.279e-12
4.42
599
,
499
1
mp mm
r
r
Compression level
Example: Updating SVDs of Random matrices
100
,
500
,
000
,
100
,
,
p
m
n
R
W
R
Y
n
m
n
p
Method
Reconstruction error
time (sec)
Strategy (A), SVD-orth
92.21
4.18
Strategy (A), QR-orth
92.21
4.23
Strategy (B)
92.21
1.93
Strategy (C)
93.41
3.54
4
.
0
,
200
,
200
rm p m m mx
r
r
The update strategies following the symmetric EVD approach are more
efficient than the SVD update known from literature (Brand, Hall et al.)
(Assumption: n>>m)
Industrial point of view: SVD/EVD performed by
‘black box function’
Most efficient: choose method depending on the compression rate
The examples suggest that all methods share a similar level of accuracy
(SVD-approach may suffer from orthogonal out-factoring,
EVD-approaches may suffer from squaring the condition number)
For details and additional references, see:
“A comprehensive comparison of various algorithms for efficiently
updating singular value decomposition based reduced order
models”,
DLR IB 124-2011/3
Freely available at DLR’s electronic library:
http://elib.dlr.de/70251