Big Data Keep it Simple

(1)

RSM Leadership Summit

Big Data – Keep it Simple

Rotterdam, October 3rd₂₀₁₄

Jens-Peter Seick, VP Head of Product Management and Development Fujitsu in Europe

(2)

1

Agenda



Big Data phenomena



Big Data technologies

(3)

A Hyperconnected World



An emerging new world where people, information, things and infrastructure

are connected via networks, transforming work and life everywhere

(4)

3

FUJITSU CONFIDENTIAL

Internet of Things & Big Data



IoT & big data bring huge growth potential to the global economy



We also face serious challenges of security and privacy

2013

10 billion

2020

50+

billion

3.6TB/h

20TB/h

100GB

- A self-driving car - A jet engine in the air - An individual genome

Things connected to the internet

FUJITSU CONFIDENTIAL

(5)

Initial Questions about Big Data

Obvious : Data is generated in large amounts

Available: Technologies for analytics

Wanted: Valuable Business ideas

What is the business idea and which information is needed? Which data is available and how can it be complemented by external sources?

What Big Data technology is appropriate and who can provide it?

(6)

5

The structure behind the applications

Sensors: Trace of the real world Feedback: Aktions in the real world Data store

 Private data store

 Online / Nearline / Archive

 Public data services

 Commercial data  … Data analysis  Cleansing  Modeling  Classification  Prediction  … Data usage  Information  Recommendation  Marketing  Product optimization  Decision  Control  … Data Sources

 Corporate Data, History

 Public Data

 Internet-Usage

 Social Networks

 Smartphone Usage

 Sensors e.g. in a car

 Quantified-Self

 …

Modeling:

Image of parts of the real world

Idea: Creating new business value Outcome: Real business value?

(7)

Manufacturing

Energy

Maintenance

Agriculture

New Opportunities for Every Industry

Big Data

Marketing

Healthcare Traffic, Transport

Public Sector

Companies from all verticals start using Big Data.

(8)

7

Example of social networks: “Patients Like Me”

 Patient groups as social networks

 Founded 2004 by Jamie Heywood

 Starting point was Amyothrophe Lateral Sklerose (ALS) disease of brother Stephen Heywood

 Patients provide information about disease symptoms and treatment results over time

 Profiting quickly form experience of others

 Trend forecasting for individual therapy adaptation und support for clinical studies.

 Currently about 250.000 members

 Profit-organization financed through providing anonymized data to industry, no advertizing

(9)

(10)

9

Be welcome in the world of Big Data

DWH / BI as it used to be  Internal  Structured  Few sources  GB and TB  Reports on history  Avoid risk  Periodic  Batch  Static model

 Few direct users

 On-premise

Today’s demands  Internal and external

 Un- / semi- / poly- / structured

 Versatile sources  TB and PB  Predict  Recognize opportunities  Ad-hoc  Real-time

 Try and innovate

 Many direct users

 Anywhere, from any device

Affordable technologies to quickly capture, store and analyze data.

Extract, Collect Transform, Cleanse Analyze, Visualize Decide, Act Extract Transform Analyze Decide

(11)

Simple Example : Quality of Wind Park Location

 Question:

 Quality of wind and solar power harvesting at certain locations?

 How develop renewable sources over time?

 Data sources:

 Weather model ERA-Interim of European Center for Medium Range Weather Forecast (ECMWF)

 Data set of >50.000 weather maps with 1 million grid points and a total volume of 25 TB

 Location based measurements

 Result:

 Generation of one million time series for grid points

(12)

11

Big Data Processing Architecture

generic model with concrete example

Consolidated data Distilled essence Applied knowledge Various data

Extract, Collect Cleanse, Transform Analyze, Visualize Decide, Act

Data Sources Analytics Platform Access

Batch processing platform Event processing platform Fast response platform Apps Services Queries Visualization Reporting Notification Data bases Application server Web content Sensor data Historical and actual weather data

Analysis for long and mid term planning

Import weather history (50.000 GRIB files)

Invert time series of maps to map of time series (1.000.000 files)

Retrieve proximate time series and calculate localized weather

ERA interim data

Visualize results

Batch platform scales out with growing data volume.

Day to day maintenance

(13)

The Missing Link: Big Data Made Easy

Data + Processing+ Presentation =

Value

!

(14)

13

Hadoop Solution at a Glance – still „big“

Complexity made easy: Get in touch with Big Data, see what is possible.

Data Sources Internal, external

Platform

Appliance, specific solution, cloud

Apps, Templates, Consulting

Insight

Visualization on any device

Generic analytics templates Templates for verticals Consulting Hadoop Rack Visual Analytics PRIMERGY Hadoop Entry Datacenter Fujitsu Cloud

(15)

Iterative Big Data Analytics Classical Business Analytics

Manage Risk, Gain Value

Inv es t / R et urn time ETL1 analysis1 operate1 h a rd w a re 1 Inv es t / R et urn time value1 _value 1 value2 value3 value4 value5 HW 1 E TL&an a ly s is1 o p e ra te1 value2

(16)

15

Example: Price Analysis for Fuel

 Question

 How does fuel pricing behave at German gas stations?

 Where is filling the cheapest?

 Data sources

 Actual retrieval of „Spritpreismonitor“

 Collecting data for several months

 Result

 Normal distribution only in the afternoon

 Massive rise in price around 19:00

(17)

Example : Quality of Weather Forecast Services

 Question

 Do weather services on Internet vary in quality?

 Which service is reliable?

 Data sources

 Web pages of multiple weather prediction services

(Wetter24, Wetterbote, myWeather, openWeather, Yahoo)

 Reference for weather observation: accuWeather

 Collecting data for European Capitals over several months

 Result

 There are significant differences

 Top are Wetter24, Wetterbote, Yahoo

(18)

17

Summary

 Big Data is a reality and creates real benefit for businesses

 Still big hurdles in understanding the value and needed competences

 New technologies for Big Data are developing fast

 Not using Big Data already starts to become a competitive threat

 Fujitsu provides infrastructure, analytic tools and services for a fast, efficient and agile solution, that keep RoI under control

(19)