Reduced Order Modeling of Geophysical Flows Using Physics-Based and Data-Driven Modeling Techniques

(1)

Reduced Order Modeling of Geophysical Flows Using Physics-based and Data-driven Modeling Techniques

By

Sk. Mashfiqur Rahman

Bachelor of Science in Mechanical Engineering Bangladesh University of Engineering and Technology

Dhaka, Bangladesh March, 2016

Submitted to the Faculty of the Graduate College of Oklahoma State University

in partial fulfillment of the requirements for

the Degree of MASTER OF SCIENCE

(2)

Reduced Order Modeling of Geophysical Flows Using Physics-based and Data-driven Modeling Techniques Thesis Approved: Omer San Thesis Advisor Arvind Santhanakrishnan Rushikesh Kamalapurkar

(3)

ACKNOWLEDGMENTS

All praises are due to Almighty for the opportunities and success I have been blessed with. I would like to thank my advisor, Dr. San for his watchful guidance, motivations, and patience. Without his valuable insights and contributions, none of this thesis work would have been possible. I would like to express my heartily gratitude towards Dr. Santhanakrishnan and Dr. Kamalapurkar for being a part of my defense committee. In addition, I would like to thank my labmates for always being supportive to me throughout the progress of my MS degree. Thanks to my family, friends, the fellow graduate students of MAE, OSU, Bangladeshi students association (BSA, OSU), respected faculties and staffs of OSU, and those who are directly or indirectly involved with this thesis work for their encouragement. Finally, I am grateful to MAE department and graduate college of Oklahoma State University for providing me the opportunity and financial assistance to pursue my MS degree in this prestigious institution. I would also like to extend thanks to Oklahoma NASA EPSCOR and the US DOE Office of Science for their financial support.

Acknowledgments reflect the views of the author and are not endorsed by committee members or Oklahoma State University.

(4)

(5)

Name: Sk. Mashfiqur Rahman Date of Degree: July, 2019

Title of Study: REDUCED ORDER MODELING OF GEOPHYSICAL FLOWS US-ING PHYSICS-BASED AND DATA-DRIVEN MODELUS-ING TECH-NIQUES

Major Field: Mechanical & Aerospace Engineering

Abstract: The growing advancements in computational power, algorithmic innovation, and the availability of data resources have started shaping the way we numerically model physical problems now and for years to come. Many of the physical phenomena, whether it be in natural sciences and engineering disciplines or social sciences, are described by a set of ordinary differential equations or partial differential equations which is referred as the mathematical model of a physical system. High-fidelity numerical simulations provide us valuable information about the flow behavior of the physical system by solving these sets of equations using suitable numerical schemes and modeling tools. However, despite the progress in software engineering and processor technologies, the computational burden of high-fidelity simulation is still a limiting factor for many practical problems in different research areas, specifically for the large-scale physical problems with high spatio-temporal variabilities such as atmospheric and geophysical flows. Therefore, the development of efficient and robust algorithms that aims at achieving the maximum attainable quality of numerical simulations with optimal computational costs has become an active research question in computational fluid dynamics community. As an alternative to existing techniques for computational cost reduction, reduced order modeling (ROM) strategies have been proven to be successful in reducing the computational costs significantly with little compromise in physical accuracy. In this thesis, we utilize the state of the art physics-based and data-driven modeling tools to develop efficient and improved ROM frameworks for large-scale geophysical flows by addressing the issues associated with conventional ROM approaches. We first develop an improved physics-based ROM framework by considering the analogy between dynamic eddy viscosity large eddy simulation (LES) model and truncated modal projection, then we present a hybrid modeling approach by combining projection based ROM and extreme learning machine (ELM) neural network, and finally, we devise a fully data-driven ROM framework utilizing long short-term memory (LSTM) recurrent neural network architecture. As a representative benchmark test case, we consider a two-dimensional quasi-geostrophic (QG) ocean circulation model which, in general, displays an enormous range of fluctuating spatial and temporal scales. Throughout the thesis, we demonstrate our findings in terms of time series evolution of the field values and mean flow patterns, which suggest that the proposed ROM frameworks are robust and capable of predicting such fluid flows in an extremely efficient way compared to the conventional projection based ROM framework.

(6)

TABLE OF CONTENTS

Chapter Page

I Introduction . . . 1

II A Dynamic Closure Modeling Framework for Model Order Reduc-tion of Geophysical Flows. . . 7

2.1 Abstract . . . 7

2.2 Introduction . . . 7

2.3 Barotropic Vorticity Equation Model. . . 13

2.4 Dynamic Closure Modeling for Reduced Order Models . . . 17

2.5 Numerical Experiments . . . 23

2.5.1 FOM simulation and data snapshots collection . . . 26

2.5.2 Experiment I: Both data collection and prediction at Re = 200, Ro = 0.0016 . . . 27

2.5.3 Experiment II: Both data collection and prediction at Re = 450, Ro = 0.0036 . . . 31

2.5.4 Experiment III: Data collection at Re = 450, Ro = 0.0036, prediction at Re = 200, Ro = 0.0016 . . . 38

2.5.5 Experiment IV: Data collection at Re = 200, Ro = 0.0016, prediction at Re = 450, Ro = 0.0036 . . . 46

2.6 Summary and Conclusions . . . 46

2.7 Appendix . . . 50

2.7.1 Time advancement scheme . . . 50

2.7.2 Numerical discretizations . . . 52

2.7.3 Numerical integration . . . 54

2.7.4 Proper orthogonal decomposition modes . . . 54

III A Hybrid Approach for Model Order Reduction of Barotropic Quasi-Geostrophic Turbulence . . . 57

3.1 Abstract . . . 57

3.3 Full Order Model (FOM) . . . 63

3.3.1 Quasi-geostrophic (QG) ocean model . . . 63

3.3.2 Numerical schemes . . . 67

3.4 Galerkin Projection Based Reduced-Order Model (ROM-GP) . . . . 68

3.5 Artificial Neural Network Based Non-Intrusive Reduced-Order Model (ROM-ANN) . . . 73

(7)

3.6 Hybrid Modeling (ROM-GP + ROM-ANN) Based Reduced-Order

Model (ROM-H) . . . 79

3.7 Numerical Results . . . 83

3.7.1 Case setup specifications for FOM simulations . . . 84

3.7.2 Analysis of the standard ROM-GP method . . . 84

3.7.3 Assessments of the prediction performance of GP, ROM-ANN, ROM-H . . . 89

3.7.4 Sensitivity analysis with respect to ELM neurons . . . 92

3.7.5 Time series evolution and out-of-sample forecasting . . . 97

3.8 Summary . . . 102

IV A Non-Intrusive Reduced Order Modeling Framework for Quasi-Geostrophic Turbulence . . . 110

4.1 Abstract . . . 110

4.3 Single-Layer Quasi-Geostrophic (QG) Ocean Circulation Model . . . 117

4.4 Intrusive ROM-GP Methodology . . . 119

4.5 Non-Intrusive ROM-LSTM Methodology . . . 123

4.6 Numerical Results . . . 128

4.7 Summary and Conclusions . . . 145

V Concluding Remarks . . . 148

5.1 Summary and Discussions. . . 148

5.2 Future Work . . . 151

(8)

LIST OF TABLES

Table Page

2.1 Numerical experiments for various (Re, Ro) combinations. . . 27

2.2 Quantitative assessments for Experiment I demonstrating the CPU time in seconds for ROM simulations (using computational time step ∆t = 2.5×10−4_{), and L}

2-norm error for the mean streamfunction

field (with respect to FOM). Note that the CPU time for the FOM simulation is about 135 hours (between t = 0 and t = 100), where computational time step is set ∆t= 2.5_×10−5_{due to the CFL restriction}

of numerical stability for our explicit forward model on the resolution of 256 × 512. Offline computing time for solving the eigensystem to find POD modes is about 21 minutes (including about 8 seconds (per 10 modes) for performing numerical integration to calculate the

(9)

2.3 Quantitative assessments for Experiment II demonstrating the CPU time in seconds for ROM simulations (using computational time step ∆t = 2.5×10−4_{), and L}

2-norm error for the mean streamfunction

field (with respect to FOM). Note that the CPU time for the FOM simulation is about 130 hours (between t = 0 and t = 100), where computational time step is set ∆t= 2.5_×10−5_{due to the CFL restriction}

of numerical stability for our explicit forward model on the resolution of 256 × 512. Offline computing time for solving the eigensystem to find POD modes is about 22 minutes (including about 8 seconds (per 10 modes) for performing numerical integration to calculate the

predetermined coefficients). Note that ∆R =R₋R˜. . . 38

3.1 A classification of reduced-order modeling approaches.. . . 61

3.2 Physics-based ROM vs. data-driven ROM. . . 81

3.3 The computational CPU time in seconds required for ROM simulations between t = 15 and t = 60. Note that the FOM simulation is 4.82 h and offline POD basis generation takes about 23.38 min. The offline training time for ANN is less than one second due to the extremely fast ELM approach. . . 93

4.1 A list of hyper-parameters utilized to train the LSTM network for all numerical experiments. . . 128

4.2 Hurst exponents of modal coefficients.. . . 131

4.3 L2-norm errors of the reduced order models (with respect to FOM)

for the mean vorticity and streamfunction fields. Note that the ROM-LSTM model trained with R = 10 modes results are presented here. . 144

(10)

4.4 Computational overhead for the ROM-LSTM model trained with R= 10 modes. For training, CPU time is presented as per epoch for 400 samples and for testing, CPU time is presented as per time step. Note that, the time step for testing is set 1.00_×10−1 _{since the non-intrusive}

(11)

LIST OF FIGURES

Figure Page

2.1 Closure modeling analogy between large eddy simulation (left) and reduced order modeling (right) where higher k index refers to smaller scales. In LES, kc and kf refer to the grid cut-off and test filtering

scales, whereas in ROM, R and ˜R indicate the number of POD modes retained in the model and test truncation process, respectively. . . 17

2.2 Graphical representation of the snapshots selection (shaded in orange) from the time histories of total kinetic energy for various (Re, Ro) combinations. . . 24

2.3 The decay of the POD eigenvalues. Note that only the first 100 largest eigenvalues (out of 400) are shown. . . 25

2.4 The relative information content showing the POD percentage energy accumulation with respect to modal index. . . 25

2.5 Some snapshots from the training data set showing the instantaneous vorticity fields. (a) Experiment I (t = 10); (b) Experiment I (t= 20); (c) Experiment I (t= 30); (d) Experiment I (t = 40); (e) Experiment II (t = 10); (f) Experiment II (t = 20); (g) Experiment II (t= 30); (h) Experiment II (t= 40). . . 28

(12)

2.6 Mean streamfunction contours for Experiment I (for training snapshots between t = 10 and t = 50). (a) FOM at a resolution of 256×512; (b) ROM-G with R = 50 modes; (c) ROM-G with R = 40 modes; (d) ROM-G with R= 30 modes; (e) ROM-G with R= 20 modes; (f) ROM-G with R= 10 modes; (g) proposed ROM-D with R= 20 modes and ∆R = 3; (h) proposed ROM-D with R= 10 modes and ∆R= 3. Note that ∆R=R−R˜. . . 33

2.7 Time series of the first modal coefficient for Experiment I. (a) ROM-G with R = 10 modes; (b) ROM-G with R= 20 modes; (c) ROM-G with R = 30 modes; (d) ROM-G with R = 40 modes; (e) ROM-G with R = 50 modes; (f) proposed ROM-D with R = 10 modes and ∆R= 3; (g) proposed ROM-D with R = 20 modes and ∆R = 3. Note that ∆R =R−R˜. True projection data is underlined in each figure with orange (training zone) and black (extended zone). . . 34

2.8 A sensitivity test with respect to the dynamic model parameter ˜R showing the time series of the first modal coefficient for Experiment I. (a) ROM-D (R = 20) with ∆R = 2; (b) ROM-D (R = 20) with ∆R = 3; (c) ROM-D (R = 20) with ∆R = 4; (d) ROM-D (R = 10) with ∆R= 2; (e) ROM-D (R= 10) with ∆R= 3; (f) ROM-D (R = 10) with ∆R= 4; (g) ROM-G with R= 20 modes. Note that ∆R =R₋R˜. True projection data is presented in each figure with orange (training zone) and black (extended zone). . . 35

(13)

2.9 A sensitivity test with respect to the dynamic model parameter ˜R showing the mean streamfunction contours for Experiment I. (a) FOM at a resolution of 256×512; (b) ROM-D (R = 20) with ∆R = 2; (c) ROM-D (R = 20) with ∆R = 3; (d) ROM-D (R = 20) with ∆R= 4; (e) ROM-G with R= 20 modes; (f) ROM-D (R= 10) with ∆R = 2; (g) ROM-D (R = 10) with ∆R= 3; (h) ROM-D (R = 10) with ∆R= 4. Note that ∆R=R−R˜. . . 36

2.10 Mean streamfunction contours for Experiment II (for training snapshots between t = 10 and t = 50). (a) FOM at a resolution of 256_×512; (b) ROM-G with R = 50 modes; (c) ROM-G with R = 40 modes; (d) ROM-G with R= 30 modes; (e) ROM-G with R= 20 modes; (f) ROM-G with R= 10 modes; (g) proposed ROM-D with R= 20 modes and ∆R = 3; (h) proposed ROM-D with R= 10 modes and ∆R= 3. Note that ∆R=R−R˜. . . 39

2.11 Time series of the first modal coefficient for Experiment II. (a) ROM-G with R = 10 modes; (b) ROM-G with R= 20 modes; (c) ROM-G with R = 30 modes; (d) ROM-G with R = 40 modes; (e) ROM-G with R = 50 modes; (f) proposed ROM-D with R = 10 modes and ∆R= 3; (g) proposed ROM-D with R = 20 modes and ∆R = 3. Note that ∆R =R₋R˜. True projection data is underlined in each figure with orange (training zone) and black (extended zone). . . 40

(14)

2.12 A sensitivity test with respect to the dynamic model parameter ˜R showing the time series of the first modal coefficient for Experiment II. (a) ROM-D (R = 20) with ∆R = 2; (b) ROM-D (R = 20) with ∆R = 3; (c) ROM-D (R = 20) with ∆R = 4; (d) ROM-D (R = 10) with ∆R= 2; (e) ROM-D (R= 10) with ∆R= 3; (f) ROM-D (R = 10) with ∆R= 4; (g) ROM-G with R= 20 modes. Note that ∆R =R₋R˜. True projection data is presented in each figure with orange (training zone) and black (extended zone). . . 41

2.13 A sensitivity test with respect to the dynamic model parameter ˜R showing the mean streamfunction contours for Experiment II. (a) FOM at a resolution of 256_×512; (b) ROM-D (R = 20) with ∆R = 2; (c) ROM-D (R = 20) with ∆R = 3; (d) ROM-D (R = 20) with ∆R= 4; (e) ROM-G with R= 20 modes; (f) ROM-D (R= 10) with ∆R = 2; (g) ROM-D (R = 10) with ∆R= 3; (h) ROM-D (R = 10) with ∆R= 4. Note that ∆R=R₋R˜. . . 42

2.14 Mean streamfunction contours for Experiment III showing the extrapo-latory predictive performance at Re = 200, and Ro = 0.0016 (i.e, using POD basis functions and mean fields associated with the training data obtained at Re = 450, and Ro = 0.0036). (a) FOM at a resolution of 256_×512; (b) ROM-G with R= 50 modes; (c) ROM-G with R = 40 modes; (d) ROM-G with R = 30 modes; (e) ROM-G with R = 20 modes; (f) proposed ROM-D with R = 20 modes and ∆R = 4; (g) proposed ROM-D with R = 20 modes and ∆R = 3; (h) proposed ROM-D with R= 10 modes and ∆R = 3. Note that ∆R =R−R˜. . 44

(15)

2.15 Time series of the first modal coefficient for Experiment III showing the extrapolatory predictive performance at Re = 200, and Ro = 0.0016 (i.e, using POD basis functions and mean fields associated with the training data obtained at Re = 450, and Ro = 0.0036). (a) ROM-G with R = 10 modes; (b) ROM-G with R= 20 modes; (c) ROM-G with R = 30 modes; (d) ROM-G with R = 40 modes; (e) ROM-G with R = 50 modes; (f) proposed ROM-D with R = 10 modes and ∆R= 3; (g) proposed ROM-D with R = 20 modes and ∆R = 3. Note that ∆R =R₋R˜. True projection data is underlined in each figure with orange (training zone) and black (extended zone). . . 45

2.16 Mean streamfunction contours for Experiment IV showing the extrapo-latory predictive performance at Re = 450, and Ro = 0.0036 (i.e, using POD basis functions and mean fields associated with the training data obtained at Re = 200, and Ro = 0.0016). (a) FOM at a resolution of 256_×512; (b) ROM-G with R= 50 modes; (c) ROM-G with R = 40 modes; (d) ROM-G with R = 30 modes; (e) ROM-G with R = 20 modes; (f) proposed ROM-D with R = 20 modes and ∆R = 4; (g) proposed ROM-D with R = 20 modes and ∆R = 3; (h) proposed ROM-D with R= 10 modes and ∆R = 3. Note that ∆R =R−R˜. . 47

(16)

2.17 Time series of the first modal coefficient for Experiment IV showing the extrapolatory predictive performance at Re = 450, and Ro = 0.0036 (i.e, using POD basis functions and mean fields associated with the training data obtained at Re = 200, and Ro = 0.0016). (a) ROM-G with R = 10 modes; (b) ROM-G with R= 20 modes; (c) ROM-G with R = 30 modes; (d) ROM-G with R = 40 modes; (e) ROM-G with R = 50 modes; (f) proposed ROM-D with R = 10 modes and ∆R= 3; (g) proposed ROM-D with R = 20 modes and ∆R = 3. Note that ∆R =R₋R˜. True projection data is underlined in each figure with orange (training zone) and black (extended zone). . . 48

3.1 Schematic representation of the extreme learning machine (ELM) neural network architecture utilized for the data-driven reduced-order modeling framework in this study. . . 79

3.2 A schematic representation of the two distinct modeling approaches: (a) physics-based modeling approach; and (b) data-driven modeling

approach. . . 80

3.3 Proper orthogonal decomposition (POD) analysis by using 900 equally distributed snapshots for Re = 25,100,400: (a) Eigenvalue spectrum of the correlation matrix, A; and (b) Eigenvalue percentage energy accumulation with respect to modal index. . . 86

3.4 Contour plots of some illustrative examples of POD basis functions for Re = 100 generated by the method of snapshots: (a) ϕ1(x, y); (b)

(17)

3.5 Mean stream function contour plots between t= 15 andt = 60 obtained by reference FOM and standard ROM-GP simulations for Re = 25: (a) ψFOM at a resolution of 128×256; (b)ψROM-GP with R= 10 modes; (c)

ψROM-GP with R= 20 modes; (d) ψROM-GP withR= 30 modes; and (e)

ψROM-GP with R = 40 modes. Note that a stable and well-estimated

solution can be found using R= 10 modes and adding more modes might yield worse solutions due to a possible over-fitting as occurred in R= 30 case. . . 88

3.6 Mean stream function contour plots betweent = 15 andt = 60 obtained by reference FOM and standard ROM-GP simulations for Re = 100: (a) ψFOM at a resolution of 128×256; b) ψROM-GP with R= 10 modes;

(c) ψROM-GP with R = 20 modes; (d)ψROM-GP with R= 30 modes; and

(e) ψROM-GP with R= 40 modes. Note that a stable and well-estimated

solution can be found within R= 30 or 40 modes for this case. . . 88

3.7 Mean stream function contour plots betweent = 15 andt = 60 obtained by reference FOM and standard ROM-GP simulations for Re = 400: (a) ψFOM at a resolution of 128×256; (b) ψROM-GP with R= 10 modes;

(c) ψROM-GP with R = 20 modes; (d)ψROM-GP with R= 30 modes; and

(e) ψROM-GP with R= 40 modes. Note that a stable and well-estimated

solution can be found within R= 40 modes for this case. . . 89

3.8 Time series evolution of the first modal coefficient,a1(t) betweent= 15

and t= 60 for FOM projection and standard ROM-GP with varying Re (Re = 25,100,400) and POD modes (R = 10,20,30,40). R = 20 modes for Re = 100, and R= 30 modes for Re = 400. . . 90

3.9 Time series evolution of the tenth modal coefficient, a10(t) between

t = 15 and t = 60 for FOM projection and standard ROM-GP with varying Re (Re = 25,100,400) and POD modes (R= 10,20,30,40). . 91

(18)

3.10 Mean stream function contour plots betweent = 15 andt = 60 obtained by reference FOM and different ROM simulations for Re = 25 and R = 10 modes: (a) ψFOM at a resolution of 128×256; (b) ψROM-GP;

(c) ψROM-ANN; and (d) ψROM-H. This figure clearly illustrates that all

models have captured the two-gyre circulation pattern successfully. . 92

(c) ψROM-ANN; and (d) ψROM-H. The figure shows that ROM-ANN and

ROM-H have captured the four-gyre circulation pattern similar to the FOM solution while ROM-GP has shown an unphysical two-gyre pattern. 93

(c) ψROM-ANN; and (d)ψROM-H. Note that only the proposed ROM-H

model has successfully captured the four-gyre circulation. . . 94

3.13 Time series evolution of the first modal coefficient,a1(t), betweent= 15

and t= 60 for FOM projection and different ROM approaches (ROM-GP, ROM-ANN, and ROM-H) with varying Re (Re = 25,100,400) and POD modes (R = 10,20,30,40). . . 95

3.14 Time series evolution of the tenth modal coefficient, a10(t), betweent =

15 andt = 60 for FOM projection and different ROM approaches (ROM-GP, ROM-ANN, and ROM-H) with varying Re (Re = 25,100,400) and POD modes (R = 10,20,30,40). . . 96

(19)

3.15 Sensitivity analysis with respect to the number of neurons in ELM of fully non-intrusive ROM-ANN approach using mean stream function contour plots (R = 10 and Re = 100): (a) ψFOM; (b) ψROM-GP; (c)

ψROM-ANN with Q= 20 neurons; (d) ψROM-ANN with Q= 40 neurons;

and (e) ψROM-ANN with Q= 80 neurons. . . 97

3.16 Sensitivity analysis with respect to the number of neurons in ELM of ROM-H approach using mean stream function contour plots (R = 10 and Re = 100): (a) ψFOM; (b) ψROM-GP; (c) ψROM-H with Q = 20

neurons; (d) ψROM-H with Q= 40 neurons; and (e)ψROM-H withQ= 80

neurons. . . 98

3.17 Sensitivity analysis with respect to the number of ELM neurons of fully non-intrusive ROM-ANN approach using time series of the first modal coefficient, a1(t) for Re = 25,100,400 and Q= 20,40,80. . . 98

3.18 Sensitivity analysis with respect to the number of ELM neurons of fully non-intrusive ROM-ANN approach using time series of the tenth modal coefficient, a10(t) for Re = 25,100,400 and Q= 20,40,80. . . 99

3.19 Sensitivity analysis with respect to the number of ELM neurons of ROM-H approach using time series of the first modal coefficient, a1(t)

for Re = 25,100,400 and Q= 20,40,80. . . 100

3.20 Sensitivity analysis with respect to the number of ELM neurons of ROM-H approach using time series of the tenth modal coefficient, a10(t)

for Re = 25,100,400 and Q= 20,40,80. . . 101

3.21 Forecasting of temporal mode evolution ofa1(t) for Re = 25. Note that

ROMs are trained by the data snapshots obtained between t= 15 and t = 60. . . 103

(20)

all ROMs are trained by the data snapshots obtained between t= 15 and t = 60. Although the ROM-GP approach provides reasonable (physical) results at the beginning (i.e., t < 18), the error then is

amplified exponentially.. . . 103

all ROMs are trained by the data snapshots obtained between t= 15 and t = 60. Although the ROM-GP approach provides reasonable (physical) results at the beginning (i.e., t < 18), the error is then

amplified exponentially.. . . 104

3.24 Time series evolution of the first temporal coefficient a1(t) for the

out-of-sample forecast by FOM and different ROM approaches (ROM-GP, ROM-ANN, and ROM-H) using R = 10 POD modes. Predictive per-formance is shown for Re = 200 while the training has been performed using the data generated at Re = 100. . . 105

3.25 Out-of-sample forecast of mean stream function by: (a) FOM; (b) ROM-GP; (c) ROM-ANN; and (d) ROM-H using R = 10 POD modes. Predictive performance is shown for Re = 200 while the training has been performed using the data generated at Re = 100. This figure clearly illustrates that the ROM-H model has successfully predicted the FOM solution while the ROM-GP model prediction is unphysical and the ROM-ANN prediction is not as accurate as the ROM-H prediction. 106

3.26 Time series evolution of the first temporal coefficient a1(t) for the

out-of-sample forecast by FOM and different ROM approaches (ROM-GP, ROM-ANN, and ROM-H) using R = 10 POD modes. Predictive per-formance is shown for Re = 200 while the training has been performed using the data generated at Re = 400. . . 107

(21)

3.27 Out-of-sample forecast of mean stream function by: (a) FOM; (b) ROM-GP; (c) ROM-ANN; and (d) ROM-H using R = 10 POD modes. Predictive performance is shown for Re = 200 while the training has been performed using the data generated at Re = 400. Similar to the previous results, this figure also shows that the ROM-H prediction is better than the other two model predictions. . . 108

4.1 Workflow diagram of the ROM-LSTM framework. Note that the train-ing phase (offline computation) is computationally heavier compared to the testing (online computation) phase. . . 126

4.2 Mean streamfunction and vorticity fields obtained by the FOM sim-ulation and the standard ROM-GP simsim-ulation at Re = 450 and Ro = 3.6_×10−3 _{flow condition. (a)} _ψ

FOM at a resolution of 256×512,

(b) ψROM-GP with R = 10 modes, (c) ψROM-GP with R = 20 modes,

(d) ψROM-GP with R = 40 modes, (e) ψROM-GP with R = 80 modes, (f)

ωFOM at a resolution of 256×512, (g) ωROM-GP with R = 10 modes,

(h) ωROM-GP with R= 20 modes, (i) ωROM-GP with R = 40 modes, (j)

(22)

4.3 Time series evolution of the first and tenth modal coefficients,a1(t) and

a10(t) respectively, between t= 10 to t = 100 for standard ROM-GP

simulation at Re = 450 and Ro = 3.6×10−3_{. (a)} _a

1(t) for ROM-GP

with R = 10 modes, (b) a10(t) for ROM-GP with R = 10 modes, (c)

a1(t) for ROM-GP with R= 20 modes, (d) a10(t) for ROM-GP with

R = 20 modes, (e) a1(t) for ROM-GP with R = 30 modes, (f) a10(t)

for ROM-GP with R= 30 modes, (g) a1(t) for ROM-GP with R= 40

modes, (h) a10(t) for ROM-GP withR = 40 modes, (i) a1(t) for

ROM-GP with R = 80 modes, (j) a10(t) for ROM-GP with R = 80 modes.

True projection series is underlined in each figure with black straight line. The training zone is shown with orange dashed line (from t= 10 to t = 50) and the out-of-sample testing zone is shown with red dashed line (fromt = 51 tot = 100) in ROM-LSTM solution series in each figure.137

4.4 Mean streamfunction and vorticity fields obtained by the ROM-LSTM simulation based on different lookback time-window,σ at Re = 450 and Ro = 3.6_×10−3 _{flow condition. (a)} _ψ

FOM at a resolution of 256×512,

(b) ψROM-LSTM with σ= 1, (c) ψROM-LSTM with σ= 2, (d) ψROM-LSTM

with σ = 4, (e) ψROM-LSTM with σ = 5, (f) ωFOM at a resolution of

256×512, (g) ωROM-LSTM with σ = 1, (h) ωROM-LSTM with σ = 2, (i)

ωROM-LSTM withσ = 4, (j)ωROM-LSTMwith σ= 5. Note that the LSTM

(23)

4.5 Time series evolution of the modal coefficients betweent= 10 tot= 100 for ROM-LSTM simulation at Re = 450 and Ro = 3.6×10−3_{. Note}

that the LSTM model is trained with R= 10 modes and σ = 5. True projection series is underlined in each figure with black straight line. The training zone is shown with orange dashed line (from t = 10 to t = 50) and the out-of-sample testing zone is shown with red dashed line (fromt = 51 tot= 100) in ROM-LSTM solution series in each figure.139

4.6 Mean streamfunction and vorticity fields obtained by the ROM-LSTM simulation based on the number of modes to train the LSTM model at Re = 450 and Ro = 3.6_×10−3 _{flow condition. (a)}_ψ

FOM at a resolution

of 256_×512, (b) ψROM-LSTM for LSTM training with R = 2 modes, (c)

ψROM-LSTM for LSTM training with R = 4 modes, (d) ψROM-LSTM for

LSTM training with R = 8 modes, (e) ψROM-LSTM for LSTM training

withR= 10 modes, (f)ωFOM at a resolution of 256×512, (g)ωROM-LSTM

for LSTM training withR= 2 modes, (h)ωROM-LSTMfor LSTM training

withR = 4 modes, (i)ωROM-LSTMfor LSTM training withR= 8 modes,

(j) ωROM-LSTM for LSTM training with R = 10 modes. Note that the

(24)

4.7 Time series evolution of the modal coefficients betweent= 10 tot= 100 for ROM-LSTM simulation based on different lookback time-windows,σ and LSTM training withR = 2 modes at Re = 450 and Ro = 3.6×10−3_.

(a) a1(t) with σ = 1, (b) a2(t) with σ = 1, (c) a1(t) with σ = 2, (d)

a2(t) with σ = 2, (e) a1(t) with σ = 3, (f) a2(t) with σ = 3, (g) a1(t)

with σ = 4, (h) a2(t) with σ = 4, (i) a1(t) with σ = 5, (j) a2(t) with

σ = 5. True projection series is underlined in each figure with black straight line. The training zone is shown with orange dashed line (from t = 10 to t= 50) and the out-of-sample testing zone is shown with red dashed line (from t= 51 to t= 100) in ROM-LSTM solution series in each figure. . . 141

4.8 Mean streamfunction and vorticity fields obtained by the ROM-LSTM simulation based on different lookback time-windows, σ and LSTM training with R= 2 modes at Re = 450 and Ro = 3.6×10−3_{. (a)}_ψ

FOM

at a resolution of 256_×512, (b) ψROM-LSTM for LSTM training with

σ = 2, (c) ψROM-LSTM for LSTM training with σ = 3, (d) ψROM-LSTM

for LSTM training with σ = 4, (e) ψROM-LSTM for LSTM training with

σ = 5, (f) ωFOM at a resolution of 256×512, (g) ωROM-LSTM for LSTM

training with σ = 2, (h)ωROM-LSTM for LSTM training with σ= 3, (i)

ωROM-LSTM for LSTM training with σ = 4, (j) ωROM-LSTM for LSTM

(25)

4.9 Time series evolution of first two modal coefficients, a1(t) and a2(t)

respectively, between t= 10 tot = 100 for different ROMs at Re = 450 and Ro = 3.6×10−3_{. (a)} _a

1(t) for ROM-GP with R = 10 modes,

(b) a2(t) for ROM-GP withR = 10 modes, (c)a1(t) for ROM-LSTM

trained withR = 2 modes, (d)a2(t) for ROM-LSTM trained withR= 2

modes, (e) a1(t) for ROM-LSTM trained withR = 4 modes, (f) a2(t)

for ROM-LSTM trained with R = 4 modes, (g)a1(t) for ROM-LSTM

trained with R = 8 modes, (h) a2(t) for ROM-LSTM trained with

R = 8 modes, (i) a1(t) for ROM-LSTM trained with R = 10 modes,

(j) a2(t) for ROM-LSTM trained withR = 10 modes. True projection

series is underlined in each figure with black straight line. The training zone is shown with orange dashed line (from t= 10 to t = 50) and the out-of-sample testing zone is shown with red dashed line (from t= 51 to t= 100) in ROM-LSTM solution series in each figure. . . 143

(26)

CHAPTER I

Introduction

Modeling of large-scale turbulent flows has been subject to an active research for over the past half century due to its impact on a variety of applications, such as ocean modeling, weather forecasting, aeronautical applications, energy harvesting and so on. The reliable prediction, control and diagnostics of the chaotic nature of these turbulence phenomena have always been a challenging task from the very beginning of turbulence studies and until now, a plethora of theoretical and computational studies have been dedicated to understand the characteristics of turbulence. High-fidelity numerical simulations like direct numerical simulation (DNS) are capable of capturing the broad range of active spatio-temporal scales in turbulent motions as the full energy spectra of turbulence can be resolved down to the Kolmogorov scale using a very fine grid resolution. Due to the continuous growth of the computer power, we now can obtain DNS of a number of simple turbulent flow problems. However, the amount of computational resources required to resolve large-scale turbulence is still unmanageable in most of the cases and even if some of the scales can be resolved, the process can turn out to be extremely inefficient and computationally expensive. It should be noted as well that the continuous progress in computer power and performance following Moore’s law has started to become saturated in the recent years. As a result, for obvious reasons, a lot of active research projects in different fields are devoted to develop efficient and robust modeling algorithms that enhance the capability of maximum attainable quality of numerical simulations with optimal computational costs. Among these, traditional computational fluid dynamics

(27)

(CFD) researches were mostly based on low-fidelity or coarse grained models by introducing modeling assumptions and often required a closure model to compensate the effects of truncated scales. Reynolds averaged Navier-Stokes (RANS) and large eddy simulation (LES) approaches are the most common coarse-grained models, although there exist many alternatives. In these approaches, the similar high-fidelity model is used with a reduced number of grid points. As an alternative to the existing traditional modeling techniques, reduced order modeling (ROM) strategies have been proven to be successful in reducing computational expenses greatly and have got a significant attention to CFD research community. In conventional snapshot-based ROM approach, the existing data snapshots, obtained from either experiments, field measurements or high-dimensional numerical simulations, are utilized to generate a lower dimensional system with an acceptable range of accuracy through a variety of mathematical operations. There are a several established ways to obtain ROM where each has their own strengths and weaknesses. Rowley and Dawson have reviewed some of the most popular ROM techniques, such as proper orthogonal decomposition (POD), dynamic mode decomposition (DMD), eigensystem realization algorithms (ERA) and so on, along with the similarities and discrepancies between them (Rowley and Dawson,

2017). In this thesis, we focus mainly on the improvement in the performance of POD based ROM by addressing their limitations using physics-based and data-driven modeling ideas.

Developing ROMs of nonlinear dynamical systems has been introduced to the com-putational fluid dynamics as a comcom-putational cost reduction approach and there have been a number of ROM techniques proposed over the years. These ROM techniques have been utilized for a wide variety of applications related to, for example, flow control, data assimilation in weather and climate modeling, uncertainty quantification, and so on. Among the different variants of ROM strategies, the Galerkin-projection combined with proper orthogonal decomposition (POD) based ROMs have become

(28)

widely popular in various areas. POD is a mathematical technique to extract the dominant statistical characteristics from a complex dynamical system by identifying the modes carrying most of the information of overall system (Lumley,1967; Benner et al., 2015; Rowley and Dawson, 2017). These few selected POD modes have the capability of representing the fine-scale details of the true physics accurately. A detailed literature review on Galerkin-projection based ROM and different updates of POD techniques are provided in the following chapters. In general, POD uses the data obtained from experiments, field measurements or high-fidelity numerical simulations and generates an orthonormal set of spatial basis vectors describing the main directions (modes) by which the flow is represented optimally (Berkooz et al.,

1993). The most energetic modes are kept to generate the reduced order system while the other modes are truncated. Indeed, there are several limitations in conventional ROM strategies, for example, getting diverging solution due to the effect of truncated modes. Often the truncated modes contribute to the evolving dynamics of complex multidimensional turbulent flows like geophysical flows. Several other issues related to POD and projection based ROM and some possible solutions proposed in different recent literature are discussed with references in the later chapters.

In this thesis work, we present three different ideas to develop an efficient and robust POD based ROM framework using three distinct type of modeling approaches: physics-based (intrusive) approach, hybrid (semi non-intrusive) approach and data-driven (fully non-intrusive) approach. To be specific, the main goal is to capture the true flow behavior using minimal amount of POD modes with optimal computational expenses. In physics-based approaches, the online ROM computation is done using some parts (or whole) of the high-fidelity model, i.e., the underlying governing equation is required to generate the ROM. In contrast, the data-driven modeling only requires the high-fidelity data snapshots to generate the ROM, i.e., no prior information about the physical system or model is required. In the data-driven approach, we replace

(29)

the physics-based Galerkin projection with any state-of-the-art data-driven tools to make the whole system fully non-intrusive or data-driven. Until recently, purely data-driven machine learning (ML) algorithms have been considered one of the most promising fields to benefit greatly from existing datasets. In the past few years, artificial neural networks (ANNs) and other ML techniques have started to make an impact in the turbulence modeling community (Raissi et al., 2019;Maulik et al.,2018;

Faller and Schreck, 1997;San and Maulik, 2018b; Wang et al., 2019; Moosavi et al.,

2015; Kani and Elsheikh, 2017). Interested readers are directed to Refs. (Brunton et al.,2019;Kutz,2017;Durbin,2018;Duraisamy et al.,2018;Gamboa,2017) for more information on ML in fluid mechanics, specifically turbulence modeling. Both physics-based and data-driven modeling strategies have some positive and negative aspects. This has led to the development of a another category of ROM framework, known as hybrid modeling approach, where both physics-based and data-driven approaches are combined in such a way that a significantly enhanced performance can be obtained than the performance of component models alone. A detailed literature review and discussion on physics-based, hybrid, and data-driven ROM techniques are presented in the following chapters.

To evaluate the performance of any ROM framework, the selection of test problem is particularly very crucial. This is because the development of ROM starts to become very challenging for large-scale turbulent flow systems with complex, nonlinear spatio-temporal scales interactions. As a result, an increased number of retained modes are required to obtain a physical and stable solution. However, increasing number of modes beyond some threshold would increase the computational cost of solving ROMs or even exceeds the cost of the original full-order model. Hence, a better ROM model should be capable of resolving the scales of such flow problems with a minimal amount of POD modes. To test our proposed frameworks, we consider the barotropic vorticity equation (BVE) representing the single-layer quasi-geostrophic (QG) ocean

(30)

circulation model as a representative of large-scale geophysical flow problems. It is well-known that much of the world’s ocean circulation is wind-driven in large-scale. Furthermore, there is an apparent connection between ocean currents and climate dynamics since the oceans are a significant part of the climate system. Due to the difference in heat capacity and density between atmospheric air and ocean water, the ocean currents are comparatively quite slow with respect to the movement of air masses in the atmosphere. The slowness of ocean current means a long timescale motions which regulate a connection between the typically shorter temporal motions of the atmosphere with the long time evolution of the climate system. It should be noted that the ocean circulation has effects on the weather as well on shorter timescales. Hence, wind-driven flows of mid-latitude ocean basins have been studied frequently by modelers using idealized single- and double-gyre wind forcing, which helps in understanding various aspects of ocean dynamics, including the role of mesoscale eddies and their effect on mean circulation (Holm and Nadiga, 2003; San et al.,2011). However, modeling the vast range of spatio-temporal scales of the oceanic flows with all the relevant physics has always been challenging. As a result, the numerical simulation of oceanic and atmospheric flows still requires approximations and simplifications of the full model. The BVE model is a simplified version of the full-fledged geophysical flow equations by considering the hydrostatic balance, geostrophic balance, theβ-plane approximation, and horizontal eddy viscosity parameterizations. To avoid complexity, we perform all the analyses in a single test problem only to quantify the predictive performance of the proposed ROM frameworks with respect to conventional projection based ROM and full order model simulation results. Throughout this document, we assess and quantify the performance of the models using mean flow fields, L2-error norms, and time series response of field values for different flow conditions. The nature and physical aspects of the wind-driven QG turbulence are detailed with references in the later chapters.

(31)

The individual chapters of this thesis apart from the introduction and conclusions are dedicated to an individual ROM development idea based on different modeling approaches. A brief summary of the core chapters of this document is presented below: 1. In chapter 2, we present a physics-based, dynamic closure modeling approach to stabilize the projection-based ROMs for the forced-dissipative QG systems. We propose an eddy viscosity closure approach to stabilize the resulting surrogate model considering the analogy between LES and truncated modal projection. Our efforts, in particular, include the translation of the dynamic subgrid-scale model into our ROM setting by defining a test truncation similar to the test filtering in LES (Rahman et al., 2019a).

2. In chapter 3, we put forth a robust hybrid ROM framework for the QG system by introducing a weighting parameter between the Galerkin projection (physics-based contribution) and extreme learning machine neural network architecture (data-driven contribution) (Rahman et al., 2018).

3. In chapter 4, we propose a fully data-driven ROM framework for large-scale QG systems which exploits the time series prediction capability of long short-term memory (LSTM) recurrent neural network architecture such that: (i) in the training phase, the LSTM model is trained on the modal coefficients extracted from the high-resolution data snapshots using POD transform, and (ii) in the testing phase, the trained model predicts the modal coefficients for the total time recursively based on the initial time history. The mean flow fields and time series response of the field values are then reconstructed from the predicted modal coefficients by using an inverse POD transform (Rahman et al., 2019b).

(32)

CHAPTER II

A Dynamic Closure Modeling Framework for Model Order Reduction of Geophysical Flows†

2.1 Abstract

In this paper, a dynamic closure modeling approach has been derived to stabilize the projection-based reduced order models in the long-term evolution of forced-dissipative dynamical systems. To simplify our derivation without losing generalizability, the proposed reduced order modeling (ROM) framework is first constructed by Galerkin projection of the single-layer quasi-geostrophic equation, a standard prototype of large-scale general circulation models, onto a set of dominant proper orthogonal decomposition (POD) modes. We then propose an eddy viscosity closure approach to stabilize the resulting surrogate model considering the analogy between large eddy simulation (LES) and truncated modal projection. Our efforts, in particular, include the translation of the dynamic subgrid-scale model into our ROM setting by defining a test truncation similar to the test filtering in LES. The a posteriori analysis shows that our approach is remarkably accurate, allowing us to integrate simulations over long time intervals at a nominally small computational overhead.

2.2 Introduction

High-fidelity numerical simulations are crucial for reliable predictions, control and diagnostics. Thanks to the huge advancement in computational power, in terms of speed (e.g., number of arithmetic operations per second) and memory, computational

(33)

fluid dynamics has witnessed substantial development during the last century. This includes using much finer numerical resolution and fewer approximations. However, there are still some situations in which computational resources cannot meet the requirements for feasible simulations. This is evident, especially in numerical weather predictions, where accurate simulations are solely applicable to regional weather models, while solution of global models is still restricted to relatively coarse grids (Kalnay, 2003; Powers et al., 2017).

Although the progress in computer power and performance has been adequately following Moore’s law (Moore, 1965, 1975) during the past decades, it is becoming increasingly obvious in worldwide semiconductor industry that it is nearing its end (Powell, 2008; Waldrop, 2016; Kumar,2018). Therefore, the development of efficient algorithms that elevate the maximum attainable quality of numerical simulations with the available resources, or at least reduce the computational cost of traditional simulations, has become a must. The latter is particularly important when multiple forward simulations are required, like those encountered in inverse problems (Daescu and Navon, 2007; Navon, 2009; Cao et al., 2007; He et al., 2011; Houtekamer and Mitchell, 1998, 2001; Bennett, 2005; Evensen, 2009; Law and Stuart, 2012; Buljak,

2011). Reduced order modeling (ROM), also known as model order reduction, is such a way of representing high-dimensional systems with much lower-dimensional (but dense) systems, resulting in substantial reduction in computational cost while keeping output quality within acceptable range (Quarteroni et al., 2015; Taira et al.,2017). This is feasible due to the fact that most high-dimensional complex systems, basically follow low-dimensional characteristic dynamics. For example, complex fluid flows often consist of superposition of spatially or temporally developing coherent structures, either growing/decaying with a specific rate, oscillating with constant frequency or containing the largest possible kinetic energy. The evolution of such structures is responsible for the bulk mass, momentum, and energy transfer. Therefore, the development of

(34)

reduced order surrogate models through extracting these underlying characteristics, would be an effective way to reduce the computational cost of numerical simulations and address more complex problems.

Among different ROM techniques, snapshot-based projection methods are particu-larly important where the time response of a system, either recorded from experiments or high-fidelity numerical simulations and given a certain input, is assumed to contain the essential behavior of that system. Proper orthogonal decomposition (POD), also known as principal component analysis, is a widely popular technique for snapshot-based ROMs (Antoulas et al.,2001a;Chinesta et al.,2011;Benner et al.,2015;Rowley and Dawson, 2017), and it has been introduced to the fluid dynamics community as a mathematical technique to extract coherent structures from turbulent flow fields (Lumley, 1967). The spatiotemporal biorthogonal decomposition, a variant of POD,

has been introduced to recover the temporal information associated with the POD modes (Aubry et al., 1991; Aubry, 1991). More recently, a spectral POD (SPOD) (Sieber et al.,2016) has been applied to extract low-dimensional compression of the full order model (FOM) extending POD to account for temporal dynamics in addition to energetic optimality. Although the spatial SPOD modes are no longer orthonormal, it was shown that the norm of the spatial modes gives further insights into the data set and reduces POD in the limiting case. A frequency domain form of POD (also called as SPOD) has been also introduced in order to account for the statistical variability of turbulent flows and a connection between their approach and resolvent analysis has been established. Loiseau et al. (Loiseau et al.,2018) have also introduced manifold modeling approach as a potentially viable compression model. They showed that a two-dimensional manifold is more accurate than an expansion with using 50 POD modes for the transient cylinder wake.

In general, POD computes a set of orthonormal basis vectors which describe the main directions (modes), by which the given dataset is characterized, in the L2 sense

(35)

(Berkooz et al., 1993). Based on the energy cascade between different modes, the most energetic POD modes are selected to generate a reduced order system. POD coupled with the Galerkin projection has been considered an efficient approach to generate ROMs for linear and nonlinear systems (Ito and Ravindran,1998; Iollo et al.,

2000;Rowley et al.,2004; Pinnau,2008; Sachs and Volkwein, 2010; Noack et al.,2017;

Puzyrev et al., 2019). It has been applied in a large number of problems involving fluid flows during the past few decades where the governing equations are projected onto these selected modes. While the high-dimensional approximation using standard discretization techniques often generates spaces with millions of degrees of freedoms, the reduced spaces spanned by ROMs are typically of order 100 or smaller (Milk et al., 2016). In practice, ROM approximation can lead to speedups of several orders of magnitude as well as great reduction in memory requirements. In fluid dynamics applications, the resulting dense system consists of triadic interactions due to the quadratic nonlinearity with an order of O(R3_{) computational load, where} _R _{refers to} the retained number of modes.

However, it has been noted that truncated modes often contribute to the evolving dynamics of complex multidimensional turbulent flows, especially encountered in geophysical systems (Lassila et al., 2014), resulting in diverging POD-based solution. This leads to either increasing the number of selected modes to better embed the underlying system, or sacrificing results quality. The latter is, of course, unacceptable and would ruin the reliability and applicability of such models. On the other hand, increasing number of modes beyond some threshold, would increase the computational cost of solving ROMs in a way that approaches, or even exceeds, the cost of the original full-order model. Moreover, Rempfer (Rempfer, 2000) has shown that a complete set of POD modes is not sufficient for a POD-Galerkin model to reproduce the full order dynamics accurately, even for many non-turbulent flows. He found that adding just small perturbations to the flow field (like those created by a small

(36)

numerical error in the integration of the ODEs) would result in a ROM that, in general, may not faithfully represent the full order model anymore, and the dynamics of the POD-Galerkin model could show instabilities while the true dynamics of the system is stable. Noack et al. (Noack et al.,2003) also reported the same observations. They proposed including an extra ’shift-mode’ that represents the shift of short-term averaged flow away from the POD space such that the Galerkin approximation also includes an accurate representation of the unstable steady solution.

Numerical instabilities due to the truncation in Galerkin models have been ad-dressed considering turbulence models (Aubry et al., 1988; Rempfer, 1994;Rempfer and Fasel, 1994b,a; Cazemier et al., 1998; Protas et al., 2015). Generally, POD modes resolve the production much better than the dissipation, leading to an excess production of turbulent kinetic energy in the POD subspace (Cordier et al., 2013). Several studies have approached this weak dissipation through the introduction of eddy-viscosity terms. Sirisup and Karniadakis (Sirisup and Karniadakis,2004) pro-posed a dissipative model based on a spectral viscosity diffusion convolution operator to improve the long-term predictions of Galerkin-based ROMs. In their approach viscosity amplitude decreases with the mode number, and it was shown to guarantee a non-oscillatory behavior except for some negligible bounded oscillations. Cordier et al. (Cordier et al., 2013) reported that constant eddy viscosity, even carefully calibrated, is nonphysical and would lead to incorrect scaling characteristics. They proposed an improved nonlinear eddy-viscosity model using data assimilation techniques (4D-Var in particular). Having a globally defined eddy viscosity coefficient might imply that the coherent structures of a high Reynolds number flow are represented by a fully resolved low Reynolds number computation. For example, ¨Osth et al. (Osth et al.¨ ,

2014) have used modal eddy viscosities to model high Reynolds number Ahmed body wake.

(37)

Galerkin projection based reduced order models, using a sufficiently small number of modes, without sacrificing much accuracy through the introduction of closure ideas (Cordier et al., 2013; Sirisup and Karniadakis, 2004; Osth et al.¨ , 2014; Couplet et al.,

2003;Kalb and Deane, 2007; Kalashnikova and Barone, 2010; Wang et al.,2011,2012;

Borggaard et al.,2011; Akhtar et al., 2012;Amsallem and Farhat, 2012; Balajewicz and Dowell, 2012; San and Iliescu,2014, 2015; Osth et al.¨ , 2014;Protas et al., 2015;

Wells et al.,2017). Dynamic eddy viscosity based closure models have been applied in large eddy simulation (LES) area to provide numerical stabilization as well as statistical fidelity preservation using an explicit test filtering procedure (Germano et al., 1991; Vreman, 2004). The filtering operation that is centred in LES has been less investigated in the ROM community. Therefore, extending state-of-the-art LES methodologies to the ROM field has been considered as a promising approach (Xie et al., 2017, 2018b,a; Mohebujjaman et al., 2019). Wang et al.(Wang et al., 2012) developed the variational multiscale and dynamic Smagorinsky models for the POD-based ROMs of structurally dominated turbulent flows, where the POD filtering has been implemented in the reconstructed space. Using dynamic LES and ROM analogy, the chief novelty of this paper is to introduce a “test truncation” mechanism, which is implemented in the reduced order space (i.e., aiming at obtaining an efficient ROM with a small computational overhead), to generate stable, self-adapted, dynamic ROMs for estimating long term dynamics of forced-dissipative turbulent flows.

To the authors’ knowledge, the application of such dynamic closure models in ROMs is very limited, and the current work is an effort for such incorporation. Following LES ideology, the eddy viscosity concept is thought to provide an efficient framework to account for the unrepresented scales due to intense mode truncation. Eddy viscosity is computed on the fly using test truncation idea, similar to test filtering in LES. To assess our idea, the barotropic vorticity equation is selected as our test bed. It is a widely used mathematical model to study the forced-dissipative large scale ocean

(38)

circulation problems, also known as the single-layer quasi-geostrophic (QG) model, first introduced by Jule Charney (Charney, 1949). We found that the proposed dynamic closure approach gives stabilized results over longer time intervals, compared with regular ROMs, with a negligible computational overhead. While presented in the vorticity-streamfunction formulation, we highlight that our modular approach can be easily generalized to the projection-based reduced order models in primitive variables.

The rest of the paper is organized as follows: Sec. 2.3 describes the governing equations briefly for the adopted test bed to generate snapshots; Sec. 2.4is devoted to the description and derivation of the dynamic eddy viscosity closure in ROM; in Sec. 2.5, we present and discuss our results of the proposed framework; and Sec. 2.6

provides a summary of this study and the conclusions drawn from it. The numerical discretization schemes used for spatial and temporal derivative approximations as well as the generation and selection criteria of POD modes are specified in the Appendix at the end of this manuscript.

2.3 Barotropic Vorticity Equation Model

Atmospheric and oceanographic flows often take place over horizontal length scales, much larger than their vertical length scale. Therefore, they can be adequately de-scribed by using the shallow water equations (SWE). The single layer two-dimensional QG equation is an approximation of the SWE, filtering the inertia-gravity waves, under the following assumptions (Vallis,2006):

• Rossby number, Ro is small, such that inertial forces are an order of magnitude smaller than the Coriolis and pressure forces,

• Horizontal scale of motion is the same order of magnitude as the deformation scale, implying that the variations in fluid depth are small compared to its total depth,

(39)

• Variations in the Coriolis parameter are small,

• The timescale is the advective timescale, hence the gravity waves, which evolve on a short timescale, are filtered out.

Much of the world’s ocean circulation is wind-driven in large-scale. Therefore, wind-driven flows of mid-latitude ocean basins have been studied by modelers using idealized single- and double-gyre wind forcing which helps in understanding various aspects of ocean dynamics, including the role of mesoscale eddies and their effect on mean circulation. The barotropic vorticity equation (BVE) describing the single-layer QG equation with dissipative and forcing terms is one of the most commonly used models for the double-gyre wind-driven geophysical flows (Majda and Wang, 2006).

The BVE model is a simplified version of the more general primitive equations used in operational weather forecast centers (Kalnay, 2003), making it a suitable model for testing new ideas. Detailed discussions on different underlying mechanisms and formulations have been presented in literature (Holland and Rhines, 1980;Munk and Wunsch,1982;Griffa and Salmon,1989;Cummins,1992;Greatbatch and Nadiga,2000;

Nadiga and Margolin, 2001). In dimensionless vorticity-streamfunction formulation, using β-plane assumption reasonable for most oceanic flows, the forced-dissipative BVE can be written as follows (Rahman et al., 2018):

∂ω ∂t +J(ω, ψ)− 1 Ro ∂ψ ∂x = 1 Re∇ 2 ω+ 1 Rosin(πy), (2.1) where_∇2 _{refers to the Laplacian in two-dimensions,}_ω_and_ψ_{are the kinematic vorticity} and streamfunction, respectively, defined as:

ω=∇ ×u, (2.2)

(40)

where u is the two-dimensional velocity field and ˆk refers to the unit vector perpen-dicular to the horizontal plane. The nonlinear advection term in Eq. (2.1) is given by the Jacobian J(ω, ψ) = ∂ψ ∂y ∂ω ∂x − ∂ψ ∂x ∂ω ∂y. (2.4)

Eq. (2.1) has two dimensionless parameters, Reynolds number, Re and Rossby number, Ro, which are related to the characteristic length and velocity scales in the following way:

Re = V L

ν , Ro = V

βL2, (2.5)

where ν is the horizontal eddy viscosity of the BVE model and β is the gradient of the Coriolis parameter at the basin center (y = 0). L is the basin length scale and

V is the velocity scale, also known as the Sverdrup velocity(Sverdrup, 1947), and is given by

V = τ0

ρH π

βL, (2.6)

where τ0 is the maximum amplitude of the double-gyre wind stress, ρ is the mean fluid density, and H is the mean depth of the ocean basin.

Despite not being explicitly represented in Eq. (2.1), there are two important relevant physical parameters, the Rhines scale, δI and the Munk scale,δM which are the nondimensional boundary layer thicknesses for the inertial and viscous (Munk) layer of the basin geometry, respectively. As a physical interpretation of these parameters in BVE model,δI accounts for the strength of nonlinearity andδM is a measure of dissipation strength. δI and δM can be defined as

δI L = V βL2 1₂ , δM L = ν βL3 1₃ (2.7)

(41)

and are related to Ro and Re by the following relations δI L = (Ro) 1 2 , δM L = Ro Re 1₃ . (2.8)

Finally, in order to satisfy the incompressibility constraint, the vorticity and stream-function are related through the following Poisson equation:

∇2_ψ ₌

−ω. (2.9)

Following (Holm and Nadiga, 2003; San et al., 2011), a four-gyre circulation problem is considered as a benchmark for oceanic flow to generate numerical data. Since ocean circulation models where the Munk and Rhines scales are close to each other, like the QG model, remain time dependent rather being converged to a steady state as time approaches to infinity (Medjo, 2000), numerical computations of these models are conducted in a statistically steady state, also known as the quasi-stationary state. Hence, in our study, we utilize numerical schemes suited for simulation of such type of ocean models and for long-time integration. In our full order model (FOM) simulations, we use a second-order accurate kinetic energy and enstrophy conserving Arakawa finite difference scheme (Arakawa, 1966). The derivatives in the linear terms are also approximated using the standard second-order finite differences. Our time advancement scheme is given by the classical total variation diminishing third-order accurate Runge-Kutta scheme (Gottlieb and Shu, 1998). Details of the numerical schemes are given in Appendix.

To close the problem, boundary and initial conditions need to be specified. Follow-ing previous studies, we use slip boundary condition for the velocity, which implies homogeneous Dirichlet boundary condition for the vorticity. Also, the impermeabil-ity boundary condition forces homogeneous Dirichlet boundary condition for the

(42)

streamfunction:

ω_|Γ =ψ|Γ= 0, (2.10)

where Γ refers to all boundary coordinates. As an initial state, we start our computa-tions from a quiescent state (i.e.,ωt=0 =ψ|t=0 = 0) and integrate the model until a statistically steady state is obtained, i.e., the wind forcing, dissipation, and Jacobian (eddy flux of potential vorticity) balance each other.

2.4 Dynamic Closure Modeling for Reduced Order Models

Test Filter Grid Cut-off Model Test Truncation POD Cut-off Model

k

E

k

c

k

f

R

Captured Modes Resolved Scales

~

Figure 2.1: Closure modeling analogy between large eddy simulation (left) and reduced order modeling (right) where higherk index refers to smaller scales. In LES, kc andkf refer to the grid cut-off and test filtering scales, whereas in ROM,R and ˜R indicate the number of POD modes retained in the model and test truncation process, respectively.

The main idea of this paper is that the effect of truncated modes can be approxi-mated dynamically. To illustrate this online ROM closure idea, we first rewrite the governing equation as ∂ω ∂t =−J(ω, ψ) + 1 Ro ∂ψ ∂x + 1 Re∇ 2 ω+ 1 Rosin(πy), (2.11) where we approximate the prognostic variable (i.e., kinematic vorticity in this case)

(43)

using the most energetic R POD modes ω(x, y, t) = ¯ω(x, y) + R X k=1 αk(t)φk(x, y), (2.12)

where ¯ω(x, y) refers to the mean vorticity field of the training data set of snapshots,

αk(t) is the kth time-dependent coefficient andφk(x, y) is the kth spatial POD mode for the fluctuating vorticity field. The derivation of such POD modes is detailed in Appendix. We note that the POD modes are orthonormal (both orthogonal and normalized), i.e.,

Z

Ω

φi(x, y)φj(x, y)dxdy =δij, (2.13) where Ω is the entire spatial domain and δij is the Kronecker delta defined by

δij =        1, if i=j, 0, if i6=j. (2.14)

To simplify our notation we use the following angle-parenthesis definition for the inner product

Z

Ω

f(x, y)g(x, y)dxdy=_hf;g_i, (2.15) and hence hφi;φji=δij. Since the vorticity and streamfunction are related through the kinematic relationship given by Eq. (2.9), we can expand the streamfunction using the same time-dependent coefficient,

ψ(x, y, t) = ¯ψ(x, y) + R X

k=1

αk(t)θk(x, y), (2.16)

where ¯ψ(x, y) refers to the mean streamfunction field of the training data set of snapshots, andθk(x, y) is the kth spatial POD mode for the streamfunction, which

(44)

can be obtained through solving the following Poisson equations (offline computing):

∇2_ψ¯₍_{x, y}_{) =}

−ω¯(x, y), (2.17)

∇2_θ_k(_{x, y}_{) =}

−φk(x, y), k = 1,2, ..., R. (2.18)

We note that the POD modes for streamfunction do not need to be orthonormal since the streamfunction is not a prognostic variable. Substituting Eq. (2.12) and Eq. (2.16) into Eq. (2.11), an orthogonal Galerkin projection is then performed by multiplying Eq. (2.11) with the spatial POD modes φk(x, y), and integrating over the entire domain Ω. The resulting dense dynamical system for αk can be written as

dαk dt =Bk+ R X i=1 Li_kαi+ R X i=1 R X j=1 Nij_kαiαj, k = 1,2, ..., R, (2.19)

where the predetermined model coefficients can be computed by the following numerical integration (offline computing)

Bk= −J(¯ω,ψ¯) + 1 Ro ∂ψ¯ ∂x + 1 Re∇ 2_ω_¯₊ 1 Rosin(πy);φk , Li_k= −J(¯ω, θi)₋J(φi,ψ¯) + 1 Ro ∂θi ∂x + 1 Re∇ 2_φ_i;_φ k , Nij_k = −J(φi, θj);φk . (2.20)

To complete the dynamical system given by Eq. (2.19), the initial conditions for αk(t) may be obtained by the following projection

αk(t0) =

ω(x, y, t0)₋ω¯(x, y);φk

, (2.21)

whereω(x, y, t0) is the vorticity field specified at initial timet0. The standard Galerkin projection ROM given by Eq. (2.19), denoted as ROM-G in this study, often yields unstable solutions when the largest R modes might not adequately capture the

(45)

system’s dynamics (Couplet et al., 2003; Kalb and Deane, 2007; Bergmann et al.,

2009; Kalashnikova and Barone, 2010;Wang et al., 2011; Carlberg et al., 2011;Wang et al., 2012;Borggaard et al., 2011; Akhtar et al., 2012; Amsallem and Farhat, 2012;

Balajewicz and Dowell, 2012; Balajewicz et al., 2013; Lassila et al., 2013; San and Iliescu, 2014, 2015; Baiges et al.,2015;Wells et al.,2017;Xie et al.,2017). As we will illustrate in our numerical examples, this is particularly true for turbulent flows. Using an eddy viscosity approach (e.g., see FIG.2.1 for the analogy between ROM and LES that is first illustrated by Akhtar et al. (Akhtar et al., 2012)), the stabilization of the ROM can be achieved by adding a regularization term to the governing equation (San and Iliescu, 2015) ∂ω ∂t =−J(ω, ψ) + 1 Ro ∂ψ ∂x + 1 Re∇ 2_ω₊ 1

Rosin(πy) +νe∇

2_ω, _(2.22)

whereνerefers to the eddy viscosity. To account for the effects of the truncated modes, following the similar Galerkin projection approach we may obtain a regularized ROM model dαk dt =Bk+ ˜Bk+ R X i=1 (Li_k+ ˜Li_k)αi+ R X i=1 R X j=1 Nij_kαiαj, (2.23) where the additional two terms can be written as

˜

Bk =hνe∇2ω¯;φki, ˜

Li_k =hνe∇2

φi;φki. (2.24)

The free stabilization parameter νe may be simply considered as a given empirical constant (Aubry et al., 1988;Wang et al., 2012). This empirical eddy viscosity idea may be improved by supposing that the amount of dissipation is not identical for all the POD modes (Rempfer, 1991;Cazemier,1997; San and Iliescu,2014). It has been, however, shown that finding an optimal value for this parameter significantly improves

(46)

the predictive performance of ROMs (San and Iliescu, 2014, 2015). Therefore, the main novelty of the present study is the derivation of an automated approach to estimate this νe parameter dynamically at each time step (online computing). An alternative dynamic determination ofνe has been presented by San and Maulik using a supervised neural network approach (San and Maulik, 2018a). However, our effort in this paper aims at developing a mathematical model based on the idea of the “test truncation”, translating the idea of the test filtering approach (Germano et al.,1991) in dynamic LES subgrid-scale models into the ROM setting.

To demonstrate our approach, we first utilize the test truncation to Eq. (2.23) considering less number of modes ˜R (i.e., ˜R < R)

dα˜k dt =Bk+ ˜Bk+ ˜ R X i=1 (Li_k+ ˜Li_k)˜αi+ ˜ R X i=1 ˜ R X j=1 Nij_kα˜iα˜j, (2.25)

where the ˜αk refers to our approximation for kth time-dependent coefficient on a test truncated space and we subtract the resulting model given by Eq. (2.25) from the original model given by Eq. (2.23) to yield the difference equation at each kth _mode

R X i=1 (Li_k+ ˜Li_k)αi+ R X i=1 R X j=1 Nij_kαiαj − ˜ R X i=1 (Li_k+ ˜Li_k)˜αi− ˜ R X i=1 ˜ R X j=1 Nij_kα˜iα˜j = dαk dt − dα˜k dt , (2.26)

and we can approximate Eq. (2.26) by the modal scale similarity hypothesis R X i=1 Li_kαi+ R X i=1 R X j=1 Nij_kαiαj− ˜ R X i=1 Li_kαi− ˜ R X i=1 ˜ R X j=1 Nij_kαiαj =₋ R X i=1 ˜ Li_kαi+ ˜ R X i=1 ˜ Li_kαi, (2.27)

(47)

using the definitions given by Eq. (2.24), we can rewrite Eq. (2.27) as Hk =νeMk, (2.28) where Hk= R X i= ˜R+1 Li_kαi+ R X i=1 R X j=1 Nij_kαiαj− ˜ R X i=1 ˜ R X j=1 Nij_kαiαj Mk=− R X i= ˜R+1 ˜ Li kαi, (2.29)

in which the predetermined coefficients of ˜Li

k can be given (offline computing) ˜

Li k =h∇

2_φ_i;_φ

ki, (2.30)

where we assume that νe is treated as constant locally in Eq. (2.24), i.e., frozen eddy viscosity field hypothesis expressed by ˜Li

k = νeL˜ik. Similar to the approach driven by Lilly (Lilly,1992) for LES, we propose a least-squares based estimation for νe in Eq. (2.28) where we define the error at each mode, Ek =Hk−νeMk. Once we square this error term

E2 k =H 2 k−2νeHkMk+νe2M 2 k, (2.31)

then the eddy viscosity coefficient in our ROM model can be computed by minimizing the sum of square errors with respect to the free model parameter νe to obtain

∂(PR˜ k=1Ek2) ∂(νe) =−2 ˜ R X k=1 HkMk+ 2νe ˜ R X k=1 MkMk. (2.32)

(48)

to give us finally the following expression for the eddy viscosity coefficient νe = PR˜ k=1HkMk PR˜ k=1MkMk , (2.33)

where Hk and Mk are computed by Eq. (2.29) at each time step (online computing). To provide always a positive eddy viscosity in our ROM simulations, we have also applied the following clipping rule (Sagaut,2006)

νe = max 0, PR˜ k=1HkMk PR˜ k=1MkMk . (2.34)

This completes the derivation of our dynamic closure model for ROM settings. We denote our proposed model as ROM-D (i.e., Eq. (2.23) equipped with Eq. (2.34)). We note that our dynamic closure framework can be applied to modal eddy viscosities (e.g., see ¨Osth et al. (Osth et al.¨ ,2014) on the need for a nonlinear subscale turbulence term in POD models) by replacing Eq. (2.32) with the modal pendant, a topic we will investigate further in future studies.

2.5 Numerical Experiments

In this section, we primarily focus on the comparative performance of the standard ROM-G model and our proposed ROM-D model in estimating the flow behavior at different flow conditions. In addition, we also assess the robustness and prediction capability of the ROM-D model through the extrapolatory prediction and sensitivity tests. To produce the ideal data set and simulation results for our evaluations, we select the single-layer QG model as our benchmark test case which has been appeared in numerous studies as test problem (Cushman-Roisin and Beckers, 2011; Holm and Nadiga,2003;San et al.,2011;Cummins,1992;Greatbatch and Nadiga,2000). Indeed, the QG test problem comes with a great challenge of capturing wide range of scales and complex flow behavior on coarse spatial resolutions, for example, resolving the