Kieran Towey October2013. Big Data, Data Analytics & GI

Kieran Towey

October2013

695,000 status updates

98,000+ tweets

698,445 Google searches

1,820TB of data created

11million instant messages

168 million+ emails sent

YouTube

Viber

Qzone Amazon Web Services GoGrid Rackspace LimeLight Jive Software salesforce.com Xactly Paint.NET Business Education Entertainment Games Lifestyle Music Navigation News

Photo & Video

Productivity Reference Social Networking Sport Travel Utilities Workbrain SuccessFactors Taleo Workday Finance box.net Facebook LinkedIn TripIt Pinterest Zynga Zynga Baidu Twitter Twitter Yammer Atlassian Atlassian MobilieIron SmugMug SmugMug Atlassian Amazon Amazon iHandy PingMe PingMe Associatedcontent Flickr Snapfish Answers.com Tumblr. Urban Scribd. Pandora MobileFrame.com Mixi CYworld Renren Xing Yandex Yandex Heroku RightScale New Relic AppFog Bromium Splunk CloudSigma cloudability kaggle nebula Parse ScaleXtreme SolidFire Zillabyte dotCloud BeyondCore Mozy Fring Toggl MailChimp Hootsuite Foursquare buzzd Dragon Diction SuperCam UPS Mobile Fed Ex Mobile Scanner Pro DocuSign HP ePrint iSchedule Khan Academy BrainPOP myHomework Cookie Doodle Ah! Fasion Girl PaperHost SLI Systems NetSuite OpSource Joyent Hosting.com Tata Communications Datapipe PPM Alterian Hyland NetDocuments NetReach OpenText Xerox Google Microsoft IntraLinks Qvidian Sage SugarCRM Volusion Zoho Adobe Avid Corel Microsoft Serif Yahoo CyberShift Saba Softscape Sonar6 Ariba Yahoo! Quadrem Elemica Kinaxis CCC DCC SCM ADP VirtualEdge Cornerstone onDemand CyberShift Kenexa Saba Softscape Sonar6 Workscape Exact Online FinancialForce.com Intacct NetSuite Plex Systems Quickbooks eBay MRM Claim Processing Payroll Sales tracking & Marketing

Commissions Database ERP CRM SCM HCM HCM PLM HP EMC Cost Management Order Entry Product Configurator Bills of Material Engineering Inventory Manufacturing Projects Quality Control SAP Cash Management Accounts Receivable Fixed Assets Costing Billing Time and Expense Activity Management Training

Time & Attendance Rostering Service Data Warehousing

The Internet

Gigabytes

Client/Server

Megabytes

Every 60 seconds

Mainframe

Kilobytes

Mobile, Social,

Big Data & The Cloud

Zettabytes

217 new mobile web users

We have gone beyond the decimal system

Information from the internet of things

10

27

This will be our digital

universe tomorrow…

Brontobyte

10

24

This is our digital universe today

= 250 trillion of DVDs

Yottabyte

10

21

1.3 ZB of network traffic

by 2016

Zettabyte

10

18

1 EB of data is created on the internet each day = 250 million DVDs worth of information.

The proposed Square Kilometer Array telescope will generated an EB of data per day

Exabyte

10

12

Terabyte

500TB of new data per day are ingested in Facebook databases

10

15

Petabyte

The CERN Large Hadron Collider

generates 1PB per second

Today data scientist uses

Yottabytes

to describe

how much government data the NSA or FBI have

on people altogether.

In the near future,

Brontobyte

will be the measurement

to describe the type of sensor data that will be generated

from the IoT (Internet of Things)

10

9

Gigabyte

10

6

A real world example:

Networks of billions of low-cost, self-powered, nano-scale sensors

CeNSE – Central Nervous System for the Earth

Airframe Integrity

Passenger Comfort

Wildlife Tracking

Personal Sensor Subnet

Large Structure Integrity

Auto Maker

Weather Service

Wildlife Research

Oil and Gas

_{Wireless Carrier}

Security Service

Airline

Department of Transportation

Retailer

Home Automation & Security

Tsunami Warning System

Real Time Traffic Conditions

Merchandise Tracking

Oil Exploration & Production

Climate Monitoring

Large sensor networks are being designed and deployed by leading HP customers

CeNSE – Central Nervous System for the Earth

The data volumes being generated

70-100,000,000,000,000 bytes per

day

25,000,000,000,000,000,000 bytes

per year

Data being generated on the back-end network

to configure, monitor and manage

1,000,000,000,000 bytes per day

A trillion sensors

by 2030, roughly 150

sensors for every person on earth

As sensor networks grow to this scale that

equates to

one million times

more

storage and processing will be required.

Variety, velocity, volume, time to value

The Challenge

48

%

Worldwide

information

volume

growth of digital

content

¹

90

%

of digital content

created by 2015 will

be

unstructured

data types

¹

75

%

of currently deployed

data warehouses

will

fail to include

unstructured data

support (by 2016)

2

_to

meet new information

velocity

and

complexity of

demands

86

%

of corporations

cannot deliver the

right information, at

right

time

to support

enterprise outcomes

all of the time

³

So why should we care??

Big Data is on the Executive Agenda

MARCH 22, 2011, 11:30 A.M. ET.

Report to the President: Every Federal Agency Needs a 'Big Data'

Strategy

February 4, 2011

How Vendors Are Lowering Big Data

Barriers

March 26, 2011

A Model for the Big Data Era

Data-centric architecture is becoming

fashionable again

Today’s Leaders Are Racing to Uncover New Value and

C-level executives recognize that data is an essential asset

and that analytical insights drive business performance

Source: Accenture research with analytics practitioners, released March 2013,

“Analytics in Action: Breakthroughs and Barriers on the Journey to ROI”

2 out of 3 appoint Chief

Analytics or Chief Data officer

High level of commitment

2 out of 3 using Predictive

Analytics to run business

Open to external partnerships

7 out of 10 are open to

external insight providers

8 out of 10 recognize that they

will have to tap external data

sources

1 out of 5 believe they

are getting the ROI

from analytics

Only 2 out of 5 believe that their

organization has access to the right

data

Clear lack of alignment to core

business processes

Data is everywhere, Analytics is everywhere !

• Customer Segmentation

• Media Benchmarking

• Marketing Return on Investment

(MROI)

• Merchandising & Assortment

Analytics

• Customer Retention & Churn

Management

• Sales Talent Analytics

• Promotions Analytics

• Pricing Analytics

• Incentives Management Analytics

• Spend Analytics

• Warranty Analytics

• Forecasting & Demand Planning

• Logistics Optimization

• Predictive Asset Maintenance

• Healthcare Compliance & Fraud

Analytics

• Banking Fraud Analytics

• PS Intelligent Processing &

Compliance Analytics

• Insurance Claims Fraud Analytics

• Talent Management Analytics

• Change Management Analytics

• Clinical Quality Workbench (Health)

• Smart Customer Operations (Resources)

• FS Risk & Regulatory Analytics (FS)

• Network Planning & Optimization (CMT)

• Biometric Analytics Solutions for PS

• Business Intelligence &

Performance Management

• Data and Content Management

• Information Integration

• Advanced Analytics Technology

Enablement

• Big Data

GI is BIG DATA, supports Analytics & will become ubiquitous

From Local