UNIFY YOUR (BIG) DATA

(1)

UNIFY YOUR (BIG) DATA

ANALYTIC STRATEGY

GIVE ANY USER ANY ANALYTIC ON ANY DATA

Scott Gnau

President, Teradata Labs

tt @t d t

[email protected]

(2)

Unify Your (Big) Data Analytic Strategy

• Technology excitement: Hadoop, MapReduce, NoSQL, in-memory

• Deliver value from big data: lower end-user and IT barriers

Bridge skills gap: achieve maximum insights and value

• Bridge skills gap: achieve maximum insights and value

• Harness value: reference architectures and use cases

March 2013 Teradata Proprietary 2

(3)

Big Data is Not a Technology

Big Data is Not a Use Case

Big Data is Not an

Architecture

Bi D t i M t Wh t i

Big Data is a Movement

Demanding More

Analytics on All Data

What is

Big Data? g

The shortest path between big

data and insight is through a

unified data architecture

(4)

Big Data Comes with BIG HEADACHES

Even free software like Hadoop is causing

companies to spend more money…Many CIOs “

believe data is inexpensive because storage has

become inexpensive. But data is inherently messy—

it can be wrong it can be duplicative and it can be

it can be wrong, it can be duplicative, and it can be

irrelevant—which means it requires handling, which

is where the real expenses come in.

”

Through 2015, 85% of Fortune 500 organizations will

be unable to exploit big data for competitive advantage.

“

Source: The Wall Street Journal. “CIOs’ Big Problem with Big Data”. Aug 2012

”

Source: Gartner. “Information Innovation: Innovation Key Initiative Overview”. April 2012

(5)

Shift from a Single Platform to an Ecosystem

"Logical" Data Warehouse

“Big Data requirements are solved by

a range of platforms including

“We will abandon the old models

based on the desire to implement for a range of platforms including

analytical databases, discovery

platforms, and NoSQL solutions

beyond Hadoop.”

based on the desire to implement for

high-value analytic applications.”

Source: “Big Data Comes of Age”. EMA and 9sight Consulting. Nov 2012.

(6)

The Evolution of Data Warehousing and BI

…But Don’t Forget the Discovery ‘Roots’

A l i f

A l i f I tI t t d t d A ti A ti Analysis of

OLTP/ERP systems Analysis of

OLTP/ERP systems

•New Tools

‘Ah !

Data Marts Data Marts

•Replicable use cases

Integrated Data Warehouse

•Enterprise Value

Active Data Warehouse

•Real time Delivery

Big Data DiscoveryBig Data Discovery

Aha Moments!!

•‘Aha!

Moments’

cases

•Differentiated Value

Value

•Business Critical

Delivery

•Production Service Levels

(7)

W H AT A R E T H E O B S TA C L E S TO B I G D ATA ?

EXISTING SKILL SETS

Make it easier to utilize

ACCELERATE ANALYTIC

Increase enterprise-wide adoption &

ACCELERATE ANALYTIC

INNOVATION

Eliminate data silo’s to use ALL data

UNIFIED DATA ARCHITECTURE

Eliminate data silo s to use ALL data

(8)

Any User Any Data Any Analysis

Unified Data Architecture for the Enterprise

Any User, Any Data, Any Analysis

Engineersg Data Scientists QuantsQ Business Analystsy

Java, C, Python, R, SAS, SQL, SQL-MapReduce, Excel, BI, Visualization, etc.

Data Discovery Integrated

Analytics & BI

SQL-H SQL-H

C t St R fi

SQL H SQL H

Capture, Store, Refine

Audio/

Video Images Text Web &

Social Machine

Logs CRM SCM ERP

(9)

Unified Data Architecture – Requirements & Benefits

TECHNICAL

REQUIREMENTS BUSINESS

BENEFITS

Integrated Analytics & BI Pervasive Intelligence

˚

• Integrated data environment

• Access by execs and analysts

• Mission-critical uptime & security

• Robust workload management

• 360˚ view of operations

• Strategic & operational intelligence

• Trusted to manage the business

• Extend throughout organization

INTEGRATED ANALYTICS & BI

Data Discovery New Iterative Insights

• Rapid exploration capabilities • Faster time to value

• Rapid exploration capabilities

• Variety of analytic techniques

• Pre-packaged analytics

• Accessible by business analysts

• Faster time to value

• Exponentially deeper insights

• More users can leverage (without hiring as many specialists)

DATA DISCOVERY

Data Staging New Data Access

• Loading, storing, and refining data in preparation for analytics

• No up front ETL required for

• Effective, low-cost technology for loading, storing, and refining data

CAPTURE | STORE | REFINE • No up-front ETL required for

loading

CAPTURE | STORE | REFINE

(10)

The Lytro and Big Data

(11)

“Interactive, Living Pictures”

(12)

See Your Business in High-Definition

• Big Analytics & Discovery Unlocks Hidden Value

Classic BI

Structured & Repeatable Analysis

“Capture only what’s needed”

IT structures the data to answer those questions IT structures the data to

answer those questions Business determines what

questions to ask

Business determines what questions to ask

IT delivers a platform for storing, refining, and analyzing all data sources

Business explores data for questions worth answering Business explores data for

questions worth answering

Big Data Analytics

Multi-structured & Iterative Analysis

Big Data Analytics

Multi-structured & Iterative Analysis

“Capture in case it’s needed”

(13)

Choose the Right Discovery Platform

3 K Q ti t A k

3 Key Questions to Ask

How many people in your organization can directly

ask big data questions?

1 How much time does it take to answer a new

question with big data?

2 Is it hard to find the right people and budget to

tackle new big data problems?

3 tackle new big data problems?

13

Need right technologies to realize business value of big data

(14)

Iterative Analytics Accelerates Discovery

Accelerate and Operationalize New & Supplemental Insights

Analytical Idea

5

Operational DB or EDW

Operationalize

or Move On Zero-ETL Data

Load/Integration

5x

Faster

^Discovery

P

Evaluate Results

SQL and non-SQL Analysis

Process Hours vs. Days

Analysis

(15)

Discovery Platform for Complete Analytics

Multi-Analytic Techniques for Exponential Value

Multi Analytic Techniques for Exponential Value

TIME SERIES ANALYSIS TEXT ANALYSIS

TIME SERIES ANALYSIS

Discover Patterns in Rows of Sequential Data

TEXT ANALYSIS

Derive Patterns and Extract Features in Textual Data

STATISTICAL ANALYSIS

High-Performance Processing of

Common Statistical Calculations

GRAPH ANALYSIS

Discover Natural Relationships p of Entities

SQL ANALYSIS

Report & Analyze Relational Data

MAPREDUCE ANALYTICS

Custom-built, domain-specific analysis

(16)

1 + 1 = 3 Unified Analytics on ALL of the data

What do you want to discover?

s Va lu e

Social AnalyticsSocial Analytics

Supply chain optimization Predictive

Event Analysis

Busines s

y

AnalyticsLog Location

Analytics

Graph Time

Series Stat-

istics ReduceMap

SQL Geo

Spatial

(17)

Network Security

Detecting network threats with MapReduce Analytics

Application Characteristics

• Millions of PCs, files and URLs translate into 100 f billi f t ti l th

Network Security Analysis

100s of billions of potential paths

• Full network detail required for analysis

• Near-real-time scoring of every network node required for fast response

Examples

• Security event management

• Zero day attack identification

• Data loss prevention

• Data packet inspection

Advanced Analytics

What this Means for Analytics?

• Recognize the pattern of activity that threat represents

• Off-line models not sufficient to keep up with

Advanced Analytics

• Constant analysis of all connections and traffic

• Interactive root cause analysis

• Rapid attack identification, validation, and response Off line models not sufficient to keep up with

quickly morphing attack behaviors

• Requires behaviorally-based system to

automate analysis, threat identification, & rapid responsep

March 2013 Teradata Proprietary

(18)

Other Examples of Unified Data & Analytics

(19)

Summary: Unified Data Architecture

Gives Any User Any Analytic on Any Data

Give business analysts in your organization the right

1 analytical tool to apply against all existing and new data ^y ^y ^g ^g

1 Unified Data Architecture™ leverages best-of-breed

technologies on the corresponding analytical problems

2

Big Data Analytics – Every company needs both a Data Warehouse and a Discovery Platform to maximize big data business value

Big Data Management – Hadoop for landing storing and refining data

3

Big Data Management Hadoop for landing, storing, and refining data

Maximize Value from ALL Your Data from Unified Architecture

(20)

Scott Gnau

President, Teradata Labs [email protected]