Big Data Analytics for Upstream Geoscience
Sunjay1, Manas Banerjee2 and Gaurav Kumar Dwivedi3 1 Dept. of Geophysics , BHU,India
2 Dept. of Geophysics , BHU,India
3 Dept. of Geophysics , BHU,India Corresponding e-mail: [email protected]
Abstract
Analytics is the discovery and communication of meaningful patterns in data. Analytics often favours data visualisation to communicate insight. Big data analytics is the use of advanced analytic techniques against very large, diverse data sets that include different types such as structured & unstructured and streaming & batch, and different sizes from terabytes, petabytes , exabytes to zettabytes. Big data can
be described by the following characteristics:
Volume ,Variety ,Velocity ,Variability ,Veracity ,Complexity. Oil and Gas Big Data with Analytics provides a complete view of upstream geoscience oil and gas industry. The acute need for optimization in the oil and gas exploration and production stages and shows how data analytics can provide such optimization with span of exploration, development, production and rejuvenation of oil and gas assets. Statistical Analysis System SAS big data analytics software to boost exploration and production. Use big data analytics to efficiently drive oil and gas exploration and production. The oil and gas industry has an opportunity to capitalize on “big data” analytics solutions. Big Data and microseismic imaging will accelerate the smart drilling oil and gas revolution. Seismic data processing ,imaging and interpretations ,modelling and simulation are integral part of subsurface deciphering with high performance computing . The Geoillustrator & interpreters focus on illustrative geology for subsurface precise imaging. During the project we will develop new and better ways to digitally model and illustrate geological data and geological concepts. By using state of the art computer graphics and interaction hardware, Geoillustrator contributes novel ideas to the science field of visualization and computer graphics. Seismic Volume Visualization for Horizon Extraction, Rapid Visualization of Geological Concepts, illustrative 3D visualization of seismic data, Flowbased segmentation of seismic data, Geoillustrator Visualisation of Rock by Graphics Processing Unit(GPU), etc. Seismic Hadoop combines Seismic Unix with Cloudera’s distribution including Apache Hadoop to make it easy to execute common seismic data processing tasks on a Hadoop(Pig) cluster.
Keywords: Big Data Analytics ,Seismic Unix, ,Hadoop Cluster , seismic imaging
Introduction
Big Data In The Oil & Gas Upstream Sector : Upstream is no stranger to Big Data. Oil & Gas companies use thousands of sensors installed in subsurface wells and surface facilities to provide continuous data-collecting, real-time monitoring of assets and environmental conditions (Brulé, 2013). The data volume is coming from sensors, spatial and GPS coordinates, weather services, seismic data, and various measuring devices. “Structured” data is handled with specific applications used to manage surveying, processing and imaging, exploration planning, reservoir modeling, production, and other upstream activities. But much of this data is
“unstructured” or “semi-structured” such as emails, word processing documents, spreadsheets, images, voice recordings, multimedia, and data market feeds, which means it’s difficult or costly to either store in traditional data warehouses or routinely query and analyze. In this case, appropriate tools for Big Data should be used (Hems & al., 2013). To support the real-time decision-making, Oil & Gas companies need tools that integrate and synthesize diverse data sources into a unified whole. Being able to process Big Data makes it possible to derive insight from the relationships that will surface when all of these sources are processed as a whole. But to unlock this value, Oil & Gas companies need
access to the appropriate technology, tools, and expertise.
Like generic Big Data, the Upstream Data is also characterized by the 6V:
Volume: Seismic data acquisition: Wide azimuth offshore seismic data acquisition ; Seismic processing
Variety: Structured: standard and data models such PPDM, SEG-Y, WITSML, PRODML, RESML, etc. ; Unstructured: images, log curves, well log, maps, audio, video, etc. ; Semi-structured: processed data such analysis, interpretations, daily drilling reports, etc. Velocity: Real-time streaming data from well-heads, drilling equipment (EDR, LWD, MWD, Mud Logging ...) and sensors (Flow, Pressure, ROP,
Veracity (Data Quality): Improve data quality ; Run integrated asset models ; Combination of seismic, drilling and production data ; Drive innovation with unconventional resources (shale gas, tight oil) ; Pre-processing to identify data anomalies Variability:Seismic Interpretation,reservoir simulation,data interpolation(sparse signal) Value: Increase speed to first oil ; Enhancing production ; Reduce costs, such as Non Productive Time (NPT); Reduce risks, especially in the areas of Helth, Safety and Environment
Theory
Exploration And Development -By combination of Big Data and advanced analytics in Exploration and Development activities, managers and experts can perform strategic and operational decision-making. The areas where the analytics tools associated with Big Data can benefit Oil & Gas exploration include:
Enhancing exploration efforts: Historical drilling and production data help geologists and geophysicists verify their assumptions in their analysis of a field where environmental regulations restrict new surveys (Feblowitz, 2012). Combine enterprise data with real-time production data to deliver new insights to operating teams for enhancing exploration efforts (Hems & al, 2013).
Assessing new prospects: Create competitive intelligence using Analytics applied to geospatial data, oil and gas reports and other syndicated feeds in order to bid for new prospects (Hems & al, 2013).
Identifying seismic traces: Using advanced analytics based on Hadoop and distributed Database for storage, quick visualization and comprehensive processing and imaging of seismic data (Seshadri, 2013) to identify potentially productive seismic trace signatures previously (Hems & al., 2013).
Figure1 : High Performance Computing With a view to high success rate of of upstream exploration and explotation business of hydrocarbon industry seismic data processing and imaging, interpretation-sterescopic rock visualization volume rendering, reservoir simulation are integral part of research and challenging task . The key challenge that the hydrocarbon industry must face for hydrocarbon exploration requires the development of state-of-the-art technologies to image subsurface precisely and reconstitute the three-dimensional structure of the Earth. High performance computing based on Graphics processing Unit (GPU)is a important direction of developments to meet the requirements of large scale computing in petroleum industry. By introducing several levels of parallelism, we can push the limit of computations and of optimization. Recent progress both on interconnect topology and new accelerating technology (FPGA, GPGPU, CELL) open new R&D directions to rectify & pacify the limits of computation. FPGA (Field-Programmable Gate Array) based neural networks implementations are increasingly used in conventional High Performance Computing applications where computational kernels such as FFT or convolution are performed on the FPGA instead of a microprocessor. The use of FPGAs for computing tasks is known as reconfigurable computing.General Purpose
Units(GPGPU) with the increasing programmability of commodity graphics processing units (GPUs), these chips are capable of performing more than the specific graphics computations for which they were designed. They are now capable coprocessors, and their high speed makes them useful for a variety of applications. FWI(Full waveform inversion ) :Acoustic, Elastic, Visco-elastic, visco-aniso-elastic
Seismic Hadoop Seismic Unix
Seismic Hadoop combines Seismic Unix with Cloudera’s distribution including Apache Hadoop to make it easy to execute common seismic data processing tasks on a Hadoop(Pig) cluster. The Apache Hadoop software library is a framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models. The massively parallel intrinsic nature of seismic data allows the geophysicist to develop very efficient algorithms.
Python seismic unix is very efficient for seismic subsurface imaging.Big Data and Extreme-scale Computing (BDEC) is
emerging field of research for high performance computing regarding seismic imaging.
Figure2 : Hadoop for SEISMIC UNIX seismic imaging
For nonstationary statistical geophysical signal analysis ,we delve deep into Principal Component Analysis, Factor Analysis, Linear Discriminant Analysis; Independent Component Analysis ,Blind Source Separation, Sparse Component Analysis by wavelet transform. Compressive sensing is a novel nonlinear sampling paradigm, effective for acquiring signals that have a sparse representation in some transform domain. The
Curse of Dimensionality is rectified by dimension reduction employing PCA,LDA etc. The acquisition modality, information processing, and inference from observations often dictates the need to deal with tensors— often big arrays of data collected in (hyper)cubes, thus generalizing the notion of data matrices. The growth of big data platforms makes it possible to solve large-scale tensor problems. High-order tensors and their decompositions are abundantly present in domains such as statistical Signal Processing (e.g., high-order moments and sensor arrays), scientific computing (e.g., discretized multivariate functions), and quantum information theory (e.g., for quantum many-body states) quantum computing . Not surprisingly, some DSPG methods result from a straightforward mapping of time series to spectral graphs, which allows for drawing parallels from the former to the latter in notions as classical as filtering, spectral analysis, and transform theory. The discrete Fourier transform (DFT) is one of SP’s “workhorses,” and its popular implementation relies on the celebrated fast Fourier transform (FFT). Recent developments, so-called sparse Fourier transform (SFT) implementation, which offers promises in certain large-scale data tasks involving sparse signals. The SFT can compute a compressed Fourier transform using only a subset of the input data in time, considerably shorter than the original data set size. SFT can thus be faster than the FFT when it is hard in large-scale applications to acquire enough data to run the FFT, and/or it is desirable to run DFT in time sublinear in the input size—a welcome attribute in medical imaging, when it is important to reduce the time that the patient spends in the magnetic resonance imaging machine. A paradigm for the analysis of graph-based data based on the so-called discrete signal processing on graphs (DSPG) approach—an effort to extend classical SP notions and techniques to data indexed by general graphs. The motivation should be clear: large data sets that are
naturally modeled as graphs are generated and analyzed in a wide range of applications, and extracting valuable information from these data requires innovative approaches.
Pattern recognition for hydrocarbon exploration
The use of pattern recognition has become more and more important in seismic oil exploration. Interpreting a large volume of seismic data is a challenging problem. Seismic reflection data in the one-shot seismogram and stacked seismogram may contain some structural information from the response of the subsurface. Syntactic/structural pattern recognition techniques can recognize the structural seismic patterns and improve seismic interpretations.The syntactic analysis methods include: (1) the error-correcting finite-state parsing, (2) the modified error-correcting Earley's parsing, (3) the parsing using the match primitive measure, (4) the Levenshtein distance computation, (5) the likelihood ratio test, (6) the error-correcting tree automata, and (7) a hierarchical system.Syntactic seismic pattern recognition can be one of the milestones of a geophysical intelligent interpretation system. Pattern Recognition: The Paragon of Big Data Analytics-there are a few minor limitations associated with predictive analytics including:Data modeling: Frequently facilitated by the paucity of data scientists, predictive analytics largely hinges on data models that are dependant on previous events to calculate the likelihood of future ones. Unexpected events may complicate if not outright evade this process, despite the fact that Machine Learning algorithms can minimize these instances.Data Types: Such models are also restricted to certain types of data and sources, whether they are structured, unstructured or semi-structured.Statistics: Predictive analytics is also circumscribed by a reliance on a linear relationship between variables, which does not take into account non-linear relationships and variables that have not been anticipated or accounted for in data models—such as which are likely to be found in Big Data sets.Touting what it refers to as its Natural Intelligence Platform, Saffron has unveiled a Cognitive Computing solution that forgoes predictive analytics for a much more profound way to identify the probability
of future events based on real-time pattern recognition of Big Data sets.
The CS-Storm system consists of multiple
high-density multi-GPU server nodes, is
well suited for HPC workloads in , oil and
gas,
pattern
recognition,
seismic
processing,
rendering
and
machine
learning
.
CONCLUSION
Now the oil and gas industry must educate “big data” on the types of data the industry captures in order to utilize the existing data in faster, smarter ways that focus on helping find and produce more hydrocarbons, at lower costs in economically sound and environmentally friendly ways. Big Data and microseismic imaging will accelerate the smart drilling oil and gas revolution. Seismic data generated in exponential manners as the upstream exploration and production cutting edge technology accelerates hydrocarbon industry to enhance R/P ratio of the nations. Seismic data processing ,imaging and interpretations ,modelling and simulation are integral part of subsurface deciphering with high performance computing . The Geoillustrator & interpreters focus on illustrative geology for subsurface precise imaging. Discrete Signal Processing on Graphs (DSPG) and Sparse Fourier Transform(SFT) are topics for lucrative creative research for Geophysical seismic signal processing. Wavelet analysis of seismic signal is integral part of big data analytics for hydrocarbon exploration and production.Nano Imaging by third generation wavelet transform is prospective for digital rock physics. Econophysics of petroleum sector time series data- Joint Organisations Data Initiative (JODI) oil & gas data plays a pivotal role in big data analytics. By using state of the art computer graphics and interaction hardware, Geoillustrator contributes novel ideas to the science field of visualization and computer graphics. Modern-day seismic-data processing, imaging, and inversion rely increasingly on computationally and data-intensive techniques to meet society’s continued demand for hydrocarbons.
We are thankful to Computer centre –BHU for providing computing & technical supports.
We are grateful to Dr. John Stockwell ,SEISMIC UNIX ,
www.cwp.mines.edu/~john
www.cwp.mines.edu/cwpcodes/ ,www.seismicunix.com I am highly obliged to Dr Lisa Gahagan Marine Seismic Data Center (MSDC)
I have login/password approved
by www.ig.utexas.edu/sdc for seismic data.
I also received Seismic data in LTO 4 tape (two) from
www.ig.utexas.edu/sdc.
Academic Seismic Portal at UTIG - Institute for Geophysics, University of Texas at Austin
Antarctic Seismic Data Library System
(SDLS)http://diam12.ogs.trieste.it/SDLS/index.php
CNSOPB Data Management Centre ,www.cnsopbdmc.ca Canada-Nova Scotia Offshore Petroleum Board REFERENCES
Baaziz A., Quoniam L., 2014, How to use Big Data technologies to optimize operations in Upstream Petroleum Industry, 21st World Petroleum Congress2014,Moscow.
Baaziz A., Quoniam L., 2013, The information for the operational risk management in uncertain environments: Case of Early Kick Detection while drilling of the oil or gas wells, International Journal of Innovation and Applied Studies (IJIAS), Vol. 4 No. 1, Sep. 2013 Brulé M., 2013, Tapping the power of Big Data for the oil and gas industry, IBM Software White Paper for Petroleum Industry, May 2013
Feblowitz J., 2012, The Big Deal About Big Data in Upstream Oil and Gas, Paper & presentation, IDC Energy Insights, October 2012
Hadoop, 2013, What Is Apache Hadoop?, Apache™ Hadoop® Website, Available at: http://hadoop.apache.org/#What+Is+Apache+Hadoop%3 F , last update: 25 August 2013
Hems A., Soofi A., Perez E., 2013, Drilling for New Business Value: How innovative oil and gas companies are using Big Data to outmaneuver the competition, A Microsoft White Paper, May 2013
Hollingsworth J., 2013, Big Data For Oil & Gas, Oracle Oil & Gas Industry Business Unit, March 2013
JDN, 2013, Panorama des solutions de BIG DATA : Hadoop et son écosystème, Journal du Net, Available at: http://www.journaldunet.com/developpeur/outils/les-solutions-du-big-data/hadoop-et-son-ecosysteme.shtml, last update : 04 July 2013
Nicholson R., 2012, Big Data in the Oil & Gas Industry, IDC Energy Insights, September 2012
Seshadri M., 2013, Big Data Science Challenging The Oil Industry, CTO Global Services, EMC Corporation, March 2013
Keith Holdaway ,2014,Harness Oil and Gas Big Data
with Analytics: Optimize Exploration and Production with Data Driven Models (Wiley and SAS Business Series),pp384
Kou-Yuan Huang ,2002,Syntactic Pattern Recognition for Seismic Oil Exploration , Series in Machine Perception and Artificial Intelligence: Volume 46 , 148pp ISBN: 978-981-02-4600-6
Georgios B. Giannakis, Francis Bach, Raphael Cendrillon, Michael Mahoney,
and Jennifer Neville ,2014 Signal Processing for Big Data
IEEE SIGNAL PROCESSING MAGAZINE SEPTEMBER 2014 PP15-16
Digital Object Identifier 10.1109/MSP.2014.2330054 Aliaksei Sandryhaila and José M.F. Moura,2014, Big
Data Analysis with Signal Processing on Graphs , IEEE
SIGNAL PROCESSING MAGAZINE , SEPTEMBER 2014 pp 80-90
Digital Object Identifier 10.1109/MSP.2014.2329213 http://www.signalprocessingsociety.org/publications/spe cial-issues-2/
http://www.signalprocessingsociety.org/publications/spe cial-issues-2/
IEEE Signal Processing Magazine
Special Issue on Geophysical Signal Processing, Part II , July 2012
http://ieeexplore.ieee.org/xpl/tocresult.jsp?isnumber=621 7361
Special Issue on Geophysical Signal Processing, Part I, May 2012
http://ieeexplore.ieee.org/xpl/tocresult.jsp?isnumber=617 9795
Signal Processing for Big Data, September 2014 http://ieeexplore.ieee.org/xpl/tocresult.jsp?isnumber=687 9573
Time-Frequency Analysis and Applications, November 2013
http://ieeexplore.ieee.org/xpl/tocresult.jsp?isnumber= 6632954#
IEEE Signal Processing ,SAM-GSSP Geophysical and seismic signal processing
IEEE Journal of Selected Topics in Signal Processing Signal Processing for Big Data , June 2015
IEEE Signal Processing ,SAM-GSSP Geophysical and seismic signal
processing ,www.signalprocessingsociety.org
https://github.com/cloudera/seismichadoop ,Seismic Hadoop combines Seismic Unix with Cloudera's Distribution including Apache Hadoop
SEISMIC UNIX SEISMIC DATA ,GPR Bore Hole Radar VRP(Vertical Radar Profiling) ,VSP (Vertical Seismic Profiling)
CWP/SU: Seismic Unix - Center for Wave Phenomena , seismicunix.com http://www.cwp. mines.edu/sututor http://www.geo.mtu.edu/~ctyoung/Robinson/Default.ht m http://www.geo.uib.no/eworkshop/index.php?n=Main. SeismicUnix
PSEIS Parallel Seismic Earth Imaging System ,http://pseis.org/
The Free Unix Seismic Processing system (FreeUSP)
http://www.freeusp.org/RaceCarWebsite/TechTransf er/Tutorials/Processing_2D/Processing_2D.html http://www.freeusp.org/RaceCarWebsite/TheToolkit/ Man_Functionality/Man_Functionality.html http://en.wikipedia.org/wiki/List_of_free_geophysics_ software http://www.freeusp.org/RaceCarWebsite/TheToolkit/ Man_Functionality/Man_Functionality.html http://www.data-forum.eu/committees ,http://www.prace-ri.eu/members/,http://www.prace-ri.eu/bdec/ http://www.deep-project.eu/deep- project/EN/Applications/SeismicImaging/seismic-cggveritas.html?nn=1163462
HPC in Oil and Gas Exploration – Prace,www.prace-project.eu,PRACE – the Partnership for Advanced Computing in Europe,http://www.adept-project.eu/case- studies/modelling-seismic-wave-propagation-for-oil-and-gas-exploration-hpc.html
Addressing Energy in Parallel Technologies
ADEPT,apos-project.eu/Articles/reservoir-simulation-in-the-oil-industry.html
www.pasc-ch.org/projects/projects/geoscale,The Kaleidoscope
Project,www.bsc.es/projects/kaleidoskope_tmp Society of High Performance Computing Professionals hpcoilgas.citris-uc.org/hpcsociety.org/ www.og-hpc.org/,http://hpcoilgas.citris-uc.org/presentations http://www.risc-project.eu, http://www.quest-
itn.org/hpc,http://www.eesi-project.eu/pages/menu/bdec.php
European Exascale Software Initiative (EESI)
http://www.exascale.org/bdec/,www.etp4hpc.eu,The European Technology Platform For High Performance Computing
ETP4HPC Association,Kaleidoscope of (Apache) Big Data Stack (ABDS) and HPC Technologies
http://hpc-abds.org/kaleidoscope/,www.exascale.org/iesp International Exascale Software Project
(IESP),http://www.eesi-project.eu/
EESI project - The European Exascale Software Initiative http://www.data-forum.eu/committees ,http://www.prace-ri.eu/members/,http://www.prace-ri.eu/bdec/ http://www.deep-project.eu/deep- project/EN/Applications/SeismicImaging/seismic-cggveritas.html?nn=1163462
www.deep-project.eu www.deep-er.eu,DEEP
(Dynamical Exascale Entry Platform). (DEEP – Extended Reach)
www.rethinkbig-project.eu, http://www.rethinkbig-project.eu/project/stakeholders
http://www.mapper-project.eu,http://www.cresta-project.eu/Collaborative Research into
ExascaleSystemware, Tools and Applications Big Data and Extreme Scale Computing challenges in Solid Earth Sciences
http://eudat.eu/ ,http://eudat.eu/eudat-etraining,EUDAT is primarily designed to provide data services for European researchers. Researchers (from academia and industry) can share, discover and re-use data via the EUDAT CDI
services,https://surfsara.nl/project/eudat , https://rd-alliance.org/groups/working-groups , Virtual Earthquake and seismology Research Community in Europe e-science environment(VERCE) , http://www.verce-project.eu/ ,
http://www.hpc.cineca.it/projects/verce,European Plate
Observing System (EPOS),VERCE: Seismic modelling,
www.ipgp.fr/~vilotte/VERCE , https://surfsara.nl/project/lofar
QUEST (QUantitative Estimation of Earth's Seismic Sources and STructure)
/http://www.quest-itn.org/http://www.epos-eu.org
SPICE Seismic wave Propagation and Imaging in Complex media: a European
network http://www.spice-rtn.org/,
https://github.com/obspy/obspy/wiki,http://geodynamics. org/cig/software/
POWERS (Parallel Oil,. Water and Gas Enhanced Reservoir Simulator)
http://www.saudiaramco.com/content/mobile/en/home/ne
ws/latest-news/2010/saudi-aramco-unveils-gigapowerstm.html?switchToMobile=1
www.hpc.kaust.edu.sa , HPC - Research Computing to Economic Development in Saudi Arabia ,
http://ese.kaust.edu.sa/www.hpc.kaust.edu.sa/http://swag. kaust.edu.sa
Earth Sciences and Engineering/The Seismic Wave Analysis Group (SWAG)
http://www-01.ibm.com/software/data/bigdata/industry-oil.html http://www.cloudera.com/content/cloudera/en/about/ hadoop-and-big-data.html http://hadoop.apache.org/ http://blog.cloudera.com/blog/2012/01/seismic-data-science-hadoop-use-case/ http://continuum.io/python-for-big-data https://www.jodidata.org/oil/support/jodi-oil-manual.aspx http://saffrontech.com/platform/ http://dst.gov.in/scientific-programme/bigdatainitiative.html http://www.ibmbigdatahub.com/infographic/four-vs-big-data/Enhancing exploration and production with big data in oil & gas/IBM Big Data for Oil and Gas
www.ibm.com/software/data/bigdata/industry-oil.html http://www.microsoft.com/enterprise/industry/manufactu ring-and-resources/oil-and-gas-mining/articles/the-appeal-of-big-data-in-oil-and-gas.aspx http://www.sas.com/storefront/aux/en/spbigdataoilgas/66 509_excerpt.pdf http://www.amazon.in/Harness-Oil-Gas-Data-Analytics/dp/1118779312 http://www.nimbix.net/articles/nimbix-accelerates-cloud- based-high-performance-data-analysis-solution-nvidia-tesla-grid-gpus/
Nimbix Accelerates Cloud-Based, High-Performance Data Analysis Solution With NVIDIA Tesla and Grid GPUs/Powerful Seismic Processing and Visualization Capabilities Address Big Data Analytics Requirements for Oil & Gas Industry
http://www.infosys.com/cloud/resource-center/Documents/big-data-spectrum.pdf http://www.wipro.com/newsletters/Winsights_Big_Data_ Key_to_Increase_Productivity_and_Competitiveness.pdf http://www.hds.com/assets/pdf/hitachi-webtech-educational-series-big-data-in-oil-and-gas.pdf http://www.cyrusone.com/pdf/key-steps-to-obtain-better-geophysical-data-analysis-and-operations.pdf https://www.icsu-wds.org/news/wds- related/international-council-for-science-icsu-and-the-challenges-of-big-data-in-science http://www.researchtrends.com/issue-30-september- 2012/international-council-for-science-icsu-and-the-challenges-of-big-data-in-science/ https://www.icsu-wds.org/news/news-archive/big-data- for-international-scientific-programmes-a-statement-of-recommendations-and-actions http://codata.org/blog/2014/06/16/big-data-for- international-scientific-programmes-a-statement-of-recommendations-and-actions/ http://www.worldscientific.com/worldscibooks/10.1142/ 4682
