Énergies renouvelables | Production éco-responsable | Transports innovants | Procédés éco-efficients | Ressources durables © 2 0 1 2 -IF P E n e rg ie s n o u v e lle s
Basin simulation for complex
geological settings
Towards a realistic modeling
P. Havé*, I. Faille, F. Willien
IFP Energies nouvelles
0 1 2 -IF P E n e rg ie s n o u v e lle s
What is « Basin simulation » ? (1/2)
Improving subsoil knowledge by modeling its geological history
Modeling from the creation of the basin to know its present state
Qualitative and quantitative information about fluids in the porous media
Reducing risks and costs of the oil exploration
Where are hydrocarbons ?
How much oil is available in a trap ?
Direction Technologie, Informatique et Mathématiques appliquées GdR MOMAS – Marseille – 15-16 octobre 2012 © 2 0 1 2 -IF P E n e rg ie s n o u v e lle s
What is « Basin simulation » ? (2/2)
Typical geological scales
Time: From 10 to 400 MY
Length : 100 km
Depth : 10 km
Highly heterogeneous media : sand, clay...
Sediment
deposition
Burying - Compaction –
Temperature increase
Cracking – expulsion
- migration
Trapping into oil
and gas fields
0 1 2 -IF P E n e rg ie s n o u v e lle s
Existing approaches : « simple geometry »
Structured grids
Conforming mesh with vertical pillars
Allows degenerated hexahedra
Parallel 3D code
Does not allow complex tectonic deformations
E.g. : fault displacements
Multi structured blocks
Sequential 2D code
Difficult to extend in 3D
Structured grid with
degenerated hexahedra
Direction Technologie, Informatique et Mathématiques appliquées GdR MOMAS – Marseille – 15-16 octobre 2012 © 2 0 1 2 -IF P E n e rg ie s n o u v e lle s
Why « Complex geological settings » ?
«««« Simple geometry »»»» areas are “
well”
known
« Complex geological » areas are promising
To push back peak oil
Recent discoveries in unexpected areas
Complex areas are difficult to understand without modelling
0 1 2 -IF P E n e rg ie s n o u v e lle s
Examples of complex geological settings (1/2)
NE
HW Cutoff of 2.3 Ma Horizon
« simple »
« complex »
Extensive setting
Direction Technologie, Informatique et Mathématiques appliquées GdR MOMAS – Marseille – 15-16 octobre 2012 © 2 0 1 2 -IF P E n e rg ie s n o u v e lle s
Faults have an important impact on flow paths
Discontinuous model
Can juxtapose stratigraphically distinct layers
Jourde et al. (2002)
Berg and Skar (2006)
« Fault zone » : thin area around faults
Can define new barriers or shorten migration path
Fault network
Huge impact on pressure field
0 1 2 -IF P E n e rg ie s n o u v e lle s
The challenge of « Complex »
A short research to market timing
2006 : first prototype
2012 : first commercial product
New 4D meshes
mixing 3D (media matrix) and 2D areas (faults)
Improved physics modeling
A wide range of geometric alterations
Fault flow : across and along
New schemes
For Stress, Darcy and Thermal models
Performance
Direction Technologie, Informatique et Mathématiques appliquées GdR MOMAS – Marseille – 15-16 octobre 2012 © 2 0 1 2 -IF P E n e rg ie s n o u v e lle s
A short research to market timing (1/2)
The choice of a C++ Framework : Arcane™
Co-developed since 2006 with CEA/DAM
Provide low level services
I/O, parallelism and network communications
Parallel data management :
Partitioning, load-balancing, checkpoint/restart...
Ensure good performance on parallel clusters
Tested with > 16k cores and > 10
9
cells
Domain decomposition parallelism : MPI, threads or hybrid of both
Task parallelism : to process a flow of (in)dependant
small
tasks
Plug-in architecture for sharing or extending base services
Mesh, Timeloop...
Numerics, Physics...
ArcaneCompletion Application exit compute-loop Application initialization
Data set & mesh
building
Arcane Startup
0 1 2 -IF P E n e rg ie s n o u v e lle s
A short research to market timing (2/2)
The choice of a C++ Framework : Arcane™
Speed up industrialization process
Debugging tools (TotalView integration, HYbrid Online Debugger for
Arcane), performance tracking, trace facilities...
High level approach :
abstract interface, embedded containers, C# binding
Supports Linux and Windows platforms
Extended for business features in ArcGeoSim™ project
Provide common facilities for research and commercial IFPEN
products (CO
2
sequestration, Enhanced Oil Recovery simulator...)
People focussed on their skills
Physics modeling
Numerical analysis
Computer Science
Integrated in a pre/post-processing environment : Open
Flow
or
or
Arcane Core
Parallelism | Load Balancing | Mesh | Module/Service Mng | I/O | Variables
Geometry
CPG | Surface | SubMesh | AMR
Numerics
Schemes | Solvers | Algorithms
Physics
Thermodynamics | Hydrodynamics | Chemistry
Business
MultiPhaseFlow | ReactiveTransport | Wells
Specific Modules Specific Services Application A rc G e o S im A p p lic a tio n Arcane Core
Parallelism | Load Balancing | Mesh | Module/Service Mng | I/O | Variables
Geometry
CPG | Surface | SubMesh | AMR
Numerics
Schemes | Solvers | Algorithms
Physics
Thermodynamics | Hydrodynamics | Chemistry
Business
MultiPhaseFlow | ReactiveTransport | Wells
Arcane Core
Parallelism | Load Balancing | Mesh | Module/Service Mng | I/O | Variables
Geometry
CPG | Surface | SubMesh | AMR
Numerics
Schemes | Solvers | Algorithms
Physics
Thermodynamics | Hydrodynamics | Chemistry
Business
MultiPhaseFlow | ReactiveTransport | Wells
Specific Modules Specific Services Application A rc G e o S im A p p lic a tio n
Specific Modules Specific Services Application
Specific Modules Specific Services Application A rc G e o S im A p p lic a tio n
Direction Technologie, Informatique et Mathématiques appliquées GdR MOMAS – Marseille – 15-16 octobre 2012 © 2 0 1 2 -IF P E n e rg ie s n o u v e lle s
0 1 2 -IF P E n e rg ie s n o u v e lle s
Workflow Temis
Flow
V3
gOcad / Kine3D
Powered by Code Aster
ArcTem
ArcTem
pre
pre
-
-
processing
processing
Mesh validation
Data preparation
Powered by Open
Flow
Eclipse based environment
ArcTem //
Direction Technologie, Informatique et Mathématiques appliquées GdR MOMAS – Marseille – 15-16 octobre 2012 © 2 0 1 2 -IF P E n e rg ie s n o u v e lle s
CPG with degenerated hexahedra
“Evolutive Mesh” : kinematic driven geometry
Faults with 3D dynamic co-refinement
AMR (2010)
Voronoï approach
IXM Format (v4 - XDMF like)
Sub-meshes (mesh view)
1D and 2D variants
0 1 2 -IF P E n e rg ie s n o u v e lle s
Mesh increments given at each geological age
Increment types:
Sediment deposition : new cells appear and grow
Compaction and kinematic deformation : nodes move
Sliding along faults : non conforming mesh and surface co-refinement
Erosion : cells disappear or may change their type
Computation mesh is interpolated between two ages
Fault = sliding surface
Direction Technologie, Informatique et Mathématiques appliquées GdR MOMAS – Marseille – 15-16 octobre 2012 © 2 0 1 2 -IF P E n e rg ie s n o u v e lle s
Improved physics and robust numerics (1/5)
Compaction and fluid flow coupling
Compaction is the engine of the fluid flow
Coupled equations
Mass conservation, Darcy law
Vertical mechanic equilibrium
Elasto-plastic rheology
Finite volume schemes
Implicit formulation
Unknowns
Cell-centered Pressure (over-pressure)
Node-centered Vertical Constraint
Cell-centered Porosity
Challenges
Robust FV schemes for « div K∇ » operator
on very messy mesh
No more vertical pillar
Faults
Faults
Faults
Faults
Fault LGR L K σ K u L u σ K n0 1 2 -IF P E n e rg ie s n o u v e lle s
Improved physics and robust numerics : Faults (2/5)
Faults
Faults
Faults
Faults
A fault is defined as a relation between 2 surfaces
The interactions between these two surfaces
have to be computed at each time step and
take into account gaps, overlaps and fault intersections
(Node→Face and Face→Face correlations)
Across fault flow
Across fault flow
Across fault flow
Across fault flow
Fault properties associated to each face
Flux through juxtaposed faces
Direction Technologie, Informatique et Mathématiques appliquées GdR MOMAS – Marseille – 15-16 octobre 2012 © 2 0 1 2 -IF P E n e rg ie s n o u v e lle s
Improved physics : Faults (3/5)
Very permeable
media
Very impermeable media
Across fault flow
Across fault flow
Across fault flow
Across fault flow
An academic test case
Increased water speed
due to connected
0 1 2 -IF P E n e rg ie s n o u v e lle s
Improved physics : Faults (4/5)
Along fault flow
Along fault flow
Along fault flow
Along fault flow
Flow parallel to the fault
Flow in a thin part of the medium (damaged zone)
Modeled using a surface flow on fault surfaces requires
Extended conservation equations
New pressure unknowns to discretize
Opposite flow surfaces connected through an across fault flow
Implemented using sub-mesh concept which defines a part of a mesh
(here codim 1) as a true mesh. Each mesh or sub-mesh has its own
connectivity.
Direction Technologie, Informatique et Mathématiques appliquées GdR MOMAS – Marseille – 15-16 octobre 2012 © 2 0 1 2 -IF P E n e rg ie s n o u v e lle s
Improved physics (5/5)
10km 40km 7kmSynthetic example
8 layers : sand and shale
8 faults
Overpressure and
water velocity
0 1 2 -IF P E n e rg ie s n o u v e lle s
Improved physics (5/5)
10km 40km 7kmSynthetic example
8 layers : sand and shale
8 faults
Velocity along fault
surfaces
Direction Technologie, Informatique et Mathématiques appliquées GdR MOMAS – Marseille – 15-16 octobre 2012 © 2 0 1 2 -IF P E n e rg ie s n o u v e lle s
Performance (1/3)
The target...
comparable
performances with legacy code on simple
geometry
and able to support complex geometry
weak scalability to aim our customers
optimized for 8-32 cores
ready for more
However, performances are a key element of the
strategy of that product.
0 1 2 -IF P E n e rg ie s n o u v e lle s
Performance (2/3)
On simple geometry...
Reference legacy code
New product
Simple geometry,
Complex geometry
complex physics,
young
physics
10 years old optimized code
An overhead due to complex data structures but
More scalable
More scalable
More scalable
More scalable
Major performance improvement will come from an
improvement of the linear solver scalability.
# procs
Durée (h:m:s)
Durée (s)
Scalabilité
Efficacité
Durée (h)
Durée (s)
Scalabilité
Efficacité
vs Visco
1
09:24:31
33871
1,0
100%
11,82
42549
1,0
100%
+26%
4
02:37:28
9448
3,6
90%
3,04
10953
3,9
97%
+16%
8
01:39:09
5949
5,7
71%
1,67
6018
7,2
88%
+1%
16
00:59:34
3574
9,5
59%
1,06
3803
11,3
70%
+6%
32
00:36:09
2169
15,6
49%
0,64
2297
18,8
58%
+6%
64
00:24:31
1471
23,0
36%
0,37
1337
32,3
50%
-9%
Direction Technologie, Informatique et Mathématiques appliquées GdR MOMAS – Marseille – 15-16 octobre 2012 © 2 0 1 2 -IF P E n e rg ie s n o u v e lle s
Performance (3/3)
Dynamic fault aware partitioning:
Zoom on two sub-domains
(own + ghost)
And more...
Dynamic partitioning facilities
Allows to address a wider range of solvers (including GPU)
Tested up to 2048 cores (robustness test)
Énergies renouvelables | Production éco-responsable | Transports innovants | Procédés éco-efficients | Ressources durables IF P E n e rg ie s n o u v e lle s
www.ifpenergiesnouvelles.fr
www.ifpenergiesnouvelles.fr
www.ifpenergiesnouvelles.fr
www.ifpenergiesnouvelles.fr