Solving Big Data Issues with Big Data Based Solutions

(1)

© Copyright 2013 Vivit Worldwide

Solving Big Data Issues with Big Data Based Solutions

(2)

© Copyright 2013 Vivit Worldwide

Hosted by

Mark Ford

Vivit Leader

Carolinas Chapter

(3)

© Copyright 2013 Vivit Worldwide

(4)

© Copyright 2013 Vivit Worldwide

Today’s Presenter

Ian Bromehead

Director, Product Marketing

HP Software

(5)

© Copyright 2013 Vivit Worldwide

Housekeeping

• This “LIVE” session is being recorded

Recordings are available to all Vivit members

• Session Q&A:

(6)

© Copyright 2013 Vivit Worldwide

Webinar Control Panel

Toggle View Window between

Full screen/window mode.

(7)

Webinar Abstract

More and more firms are trying to determine how they can best-use the mountains and mountains of information in their

possession.

Banks need faster data processing and sophisticated analytics to achieve a 360-degree view of the customer,

developing true relationship-based pricing, and achieve critical objectives for banking transformations such as a

customer-focused enterprise, optimized enterprise risk management, and more flexible and streamlined operations

At one of the biggest retailers in N. America, customer analyses and market trends taking 30days, were reduced to

3hours using Big Data. Not only that, but now they are able to launch marketing campaigns tailored to buying patterns,

customer demographics and event.

Healthcare institutions have used Big Data based solution to execute healthcare data analytics and provide workflow

information that helps the entity to customize business services for individual physicians, and also helps the company

develop educational materials to help train physicians on the most effective

Big data projects that are making city government smarter and more efficient make it easier to spot patterns and pull

meaning from information such as sensors and video feeds that can’t be neatly organized in a spreadsheet. Federal

entities are exploring how big data can democratize scientific research and better spot waste, fraud and abuse.

How do new Big Data capabilities turn into tangible value? Is the business value of Big Data being able to do something

faster today than yesterday? Solving Big Data issues, actually requires a Big Data solution.

What is common to all of these examples apart from Big Data, is that IT should innovate to provide agile means to

support the business, whatever means the business prefers to choose to facilitate delivery to their customers.

In this session we describe some of the above examples, and tell you how HP Software is providing leading, innovative

Big Data based solutions to help IT in this innovation.

(8)

Agenda

• Big Data trends today

• Adoption and application in the industry

• Is important to manage Big Data Resources

• How HO can help

• Applying Big Data to IT Operations

• Call to Action

(9)

Accelerating innovation & time to value

695,000 status updates

98,000+ tweets

698,445 Google searches

1,820TB of data created

11million instant messages

168 million+ emails sent

YouTube

Viber

Qzone Amazon Web Services GoGrid Rackspace LimeLight Jive Software salesforce.com Xactly Paint.NET Business Education Entertainment Games Lifestyle Music Navigation News

Photo & Video

Productivity Reference Social Networking Sport Travel Utilities Workbrain SuccessFactors Taleo Workday Finance box.net Facebook LinkedIn TripIt Pinterest Zynga Zynga Baidu Twitter Twitter Yammer Atlassian Atlassian MobilieIron SmugMug SmugMug Atlassian Amazon Amazon iHandy PingMe PingMe Associatedcontent Flickr Snapfish Answers.com Tumblr. Urban Scribd. Pandora MobileFrame.com Mixi CYworld Renren Xing Yandex Yandex Heroku RightScale New Relic AppFog Bromium Splunk CloudSigma cloudability kaggle nebula Parse ScaleXtreme SolidFire Zillabyte dotCloud BeyondCore Mozy Fring Toggl MailChimp Hootsuite Foursquare buzzd Dragon Diction SuperCam UPS Mobile Fed Ex Mobile Scanner Pro DocuSign HP ePrint iSchedule Khan Academy BrainPOP myHomework Cookie Doodle Ah! Fasion Girl PaperHost SLI Systems NetSuite OpSource Joyent Hosting.com Tata Communications Datapipe PPM Alterian Hyland NetDocuments NetReach OpenText Xerox Google Microsoft IntraLinks Qvidian Sage SugarCRM Volusion Zoho Adobe Avid Corel Microsoft Serif Yahoo CyberShift Saba Softscape Sonar6 Ariba Yahoo! Quadrem Elemica Kinaxis CCC DCC SCM ADP VirtualEdge Cornerstone onDemand CyberShift Kenexa Saba Softscape Sonar6 Workscape Exact Online FinancialForce.com Intacct NetSuite Plex Systems Quickbooks eBay MRM Claim Processing Payroll Sales tracking & Marketing

Commissions Database ERP CRM SCM HCM HCM PLM HP EMC Cost Management Order Entry Product Configurator Bills of Material Engineering Inventory Manufacturing Projects Quality Control SAP Cash Management Accounts Receivable Fixed Assets Costing Billing Time and Expense Activity Management Training

Time & Attendance Rostering Service Data Warehousing

The Internet

Gigabytes

Client/Server

Megabytes

Every 60 seconds

IBM Unisys Burroughs Hitachi NEC Bull Fijitsu Mainframe Kilobytes

Mobile, Social,

Big Data & The Cloud

Zettabytes

217 new mobile web users

(10)

Big Data Market Trends

Big data market trends

• IDC FY13 Market Spending: $10 Billion, CAGR 40% , 30%

Software

• Big data pure play vendors : $0.5 Billion FY12

• Vertica (NO.1), Opera, Aster Data , Splunk,

Greenplum

• Traditional players : Teradata, Oracle, IBM, EMC, SAP etc.

• IDC Hadoop-Map reduce software ecosystem FY13

• $378 Million , CAGR : 60%

VERTICA - Next Generation Data Warehouse

HADOOP - Open Source for Big Data

(11)

There is a huge opportunity

Relational

Technology

(1990-2000)

$100B

SAP, Oracle, Siebel,

PeopleSoft...

Growth

Information monetized

100 x!

10%

100%

n-Apps

Real time

info

processing

10x

(12)

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 12 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Leverage your data, make it matter

Innovative companies are changing rules of the

game

• Develop disruptive business models

• Create better products and services

• Enhance customer experience

• Drive sustained competitive

advantage

(13)

Innovative analytic use cases are cutting across structured, unstructured and semi structured data

Big Data opportunities across industries and use cases

Government

Telecom

Manufacturing

Healthcare

• Sentiment analysis

• Social CRM / network analysis

• Churn mitigation

• Brand monitoring

• Cross and Up sell

• Loyalty & promotion analysis

• Web application optimization

• Marketing campaign optimization

• Brand management

• Social media analytics

• Pricing optimization

• Internal risk assessment

• Customer behavior analysis

• Revenue assurance

• Logistics optimization

• Clickstream analysis

• Influencer analysis

• IT infrastructure analysis

• Legal discovery

• Equipment monitoring

• Enterprise search

• Drug

development

• Scientific research

• Evidence based

medicine

• Healthcare

outcomes

analysis

• Supply chain

optimization

• Defect tracking

• RFID Correlation

• Warranty

management

• Broadcast

monitoring

• Churn prevention

• Advertising

optimization

• Law enforcement

• Counter terrorism

• Traffic flow

optimization

Horizontal use cases

Sources: IDC: 2012 “Worldwide Big Data Technology and Services Forecast: 2011-2015, Gartner: 2012 “Big Data Drives Rapid Changes in Infrastructure and $232 Billion in IT Spending

Through 2016

Finance

• Fraud detection

• Anti-money

laundering

• Risk management

Energy

• Weather

forecasting

• Natural resource

exploration

(14)

Large USA appliance reseller 2600 stores

Large retailer using BigData

Benefits

-

Reports in 3 days instead of 6-12 weeks

-

Costs 33% of legacy RDBMS systems

Marketing

Campaign

Campaign1

Campaign2

Campaign 3

30

3 #

Of

Days

Before After

BEFORE BigData

AFTER BigData

Customer Analysis

100%

10%

Data Window

Examined Before

Data Window

Examined After

Benefits

-

Mobile commerce analyses done daily not weekly

-

1 min difference between 100M and 2B records

(15)

One platform for structured, semi, and unstructured to profit from 100% of data.

Big Data needs a unified approach

• Capture

• Store

• Manage

• Analyze

• Optimize

Universal log management

Structured warehouses

Unstructured

100% of data

Enable you to:

on

CRM, transactions, sales, marketing…

IT logs, security logs, social, tweets, JSOn’s

(16)

HAVEn – Big Data platform

HAVEn

Social media

Video

Audio

Email

Texts

Mobile

Transactional

_data

IT/OT

Search engine

Images

Catalog massive

volumes of

distributed data

H

adoop/

HDFS

Process and

index all

information

A

utonomy

IDOL

Analyze at

extreme scale

in real-time

V

ertica

Collect & unify

machine data

E

nterprise

Security

Powering

HP Software

+

your

apps

n

_Apps

Documents

hp.com/haven

(17)

Big Data Solutions for IT

Key IT challenges

(18)

Mobility

Today’s IT

Point tool proliferation for management

Software-defined datacenter

Optimized fabric + networking

On-premise + cloud

Bring your own device

Composite apps

(19)

April 16 2013

Cause of the problem

“Not immediately clear”

$100 million

No single

cause

Perfect

storm:

(20)

• Amazon.com offline 30 to 40 minutes,

estimated $3 million in losses

1 • Outlook.com users locked out up to 7

hours

1 • Google Search, Gmail,YouTube stopped

working worldwide

2 • American Express out of service for

hours

3 • 32k users incapacitated

at South Korea's

banks and television networks

3 1 – Evolven.com, 2 –

Minyanville

, 3 -

UsaToday

Problem:

How do you defend

against the perfect

storm?

(21)

Gather and retain

all the data

Store it in a

system that

understands what

it means

System Events

Log files

Machine Data

Telemetry

Business metrics

Structured

Unstructured

LOB

Solution:

Prepare

(22)

Harness the

power of

Predictive

Analytics

Uncover trends

before they

become

catastrophes

Analytics

Extract signal

from the noise

Spot relationships

Machine

learning

Subtle effects

Forecasting

Solution:

Predict

(23)

Solution:

Pinpoint

Replay history to

recreate the

events

Pinpoint root

causes with

advanced

analytics

Haystack

On a rocket

In a storm

Needle

(24)

Prepare

Big Data store for

IT

Connects to all your

operations data

Stores, indexes and

understands

Automated

Analytics

Self-calibrates via

machine learning

Predicts future

behavior

Instant replay

Recreates history

with beautiful

visualizations

Isolates trouble in

time…in context

Predict

Pinpoint

Solution:

HP Operations Analytics

(25)

80%

safer

Reduced severe outage

incidents

80%

faster

To diagnose complex

trouble

80%

lower cost

Than 3

rd

_{party solutions}

ROI: we hope you’ll never find out

HP Operations Analytics

(26)

(27)

Featuring the Run Time Service Model

HP Business Service Management

Ensure optimal IT performance and availability…in a dynamic world

Business Service Management

Forrester &

Gartner Leader

Ops Analytics the

next big thing!

Unique RTSM

engine

Patented

Analytics: IT’s

crystal ball

Huge Installed

Base

Operations

Intelligence

Actionable analytics from

events, logs, metrics & topology

data

Comprehensive visibility

and monitoring of hybrid

IT

Infrastructure Management

●●

Consolidated management,

accelerated root cause

analysis

Operations Bridge

End to end composite

application performance

management

Application

Management

(28)

Comprehensive, actionable insight into all aspects of operations for the New Style

of IT

HP Operations Analytics

Application

Mobile app

Storage

Network

System

Cloud

Standalone, Scalable Operations Analytics platform

Operational

Events

Performance

Metrics

Topology

Logs

Guided Troubleshooting

IT Search

Visual Analytics

3 rd

_{Party tools e.g. Splunk}

New

standalone

solution

(29)

Guided Troubleshooting

Visual Analytics

Phrase Query

Language

(PQL)

Analytics Query

Language

(AQL)

Provides Advanced, Big Data Analytics for the New Style of IT

HP Operations Analytics

IT Search

Analytics Query

Language

(30)

For whom and why?

Persona

Primary use cases

Subject Matter Experts

-Application Support

Infra/Network Support

• Application & Infra problem isolation using Guided

Troubleshooting with application specific content &

visualization

• IT Search – Ad hoc use cases for problem isolation

• Problem analysis against event data, logs and metrics to

understand repeat issues, or service delivery improvements

Data Analyst

Data specialist

• The ability to create ad hoc queries against metrics, events,

topology and logs to gain insight with analytics tools such as

trending, volume and custom algorithms

Problem Manager

Service Manager

NOC/Ops Bridge Mgr

(31)

Why care about ‘HP Operations Analytics’

Why HP for HP Operations Analytics?

• With HP Operations Analytics

• Prepare, Predict, Pinpoint

• 80% Faster - practice analytics otherwise humanly not feasible

• 80% Safer – less downtime, reduce triage time, reduce outages

• 80% cheaper – costs not related to data volume collected

• Embeds Vertica, state of the art Big Data technologies

• Embeds Arcsight Logger, leader in Gartner MQ Quadrant

• HP Operations Analytics

• Is Enterprise ready and scalable to virtually any store size

• Protects your investment

• Exploits existing data from HP data collectors incl EUM

• Integrates but is independent of HP BSM

• Integrates with 3

rd

_{Party solutions e.g. Splunk}

• voted Strong Value by EMA

“As both a platform and a vision, HP

Operations Analytics offers a highly modular

and adaptive approach to tackling new and

even unforeseen problem sets.

HP’s introduction of Operations Analytics has

all the makings of a visionary industry event.”.

Dennis Drogseth, Vice President

(32)

HP Operations Analytics

Explore 1

st

_{Generation operations}

analytics

-

Download the freemium integration

components

-

(Optional) Download freemium Arcsight logger

and/or OMi

-

Integrate your OMi or OM or NNMi

-

Exploit Award winning search to find issues

fast

Adopt the New Style of IT

Management

-

Ask to enter the Early Release Program for HP

Operations Analytics

-

Install the Virtual appliance

-

Get all the advantages of BigData Analytics for

IT Operations

(33)

“At Vodafone Ireland, we see our successful

implementation of HP BSM software for consolidated

operations, being extremely well complemented by the

potential we see in HP Operations Analytics.

The capability to rapidly search through massive

amounts of data, events, topology, metrics, machine

data and log files, will complement our monitoring

solution, and helps us master the dynamics of our

Hybrid IT environment”

_{Luke Bradley, Service Management Tools}

Manager, Technology Operations,

Vodafone Ireland

(34)

Predictive Analytics for IT

Operations

(35)

No early warning of service

degradation

No easy way to correlate if

current issue is similar to

those seen in the past

Static thresholds are

problematic to maintain

And sometimes impossible

to set

Enterprise IT environments

are more diverse, complex

and constantly changing

(36)

HP BSM: Service Health Analyzer

• Predictive alerting based on learned

pattern detection

• Unique topology-based performance analytics

• Automated root-cause and similarity analysis

• HP patented algorithms / IP

Foresight

of potential issues with behavioural learning and predictive analytics

“SHA Detected an issue 36 minutes before

regular monitoring detected it”

Healthcare Provider

“SHA detected an issue 1 hour before it was

reported by end user”

(37)

SHA Value Summary

Reduce

negative

business

Impact

 Get notified earlier

 Prevent and reduce business impact

of problems

Reduce Labor

Costs

 No need to set and maintain

thresholds

 Triage multiple events at the same

time

 Avoid reinventing the wheel -

automatically detect similar issues

Low Risk

Deployment

 Leverage BSM investment and RTSM

Topology

 Analyze any time series data

 Easy to deploy

Product

St

rengt

h

Cost Efficiency

(38)

Predictive Analytics with HP SHA

Attend the HP Virtual Events

September 10, 2013 | Americas

October 1, 2013 | EMEA

Follow the Webinar

Get the power of predictive analytics featuring

Forrester Research

Read the Solution Brief

You can anticipate IT issues before they impact your business.

Predict and preemptively solve more IT problems with HP

Service Health Analyzer

Watch the HP Service Health Analyzer

overview and demo

Learn how customers have gained

success

(39)

The management cockpit for Big Data

platforms such as Hadoop and Vertica

“As a Hadoop Administrator, I would like to

know the availability of the Hadoop

(40)

HP Operations Manager i + Management Packs

HP Big Data Monitoring Cockpit

Real time

Visualization

of your

end to end big data environment

Monitor

availability &

performance of big data

applications & infrastructure

components

Isolate, diagnose &

Remediate

problems impacting your

complex big data environment

Single version of the

truth for all Operations

Centralise management

information, faster triage

Lowered costs, increaed

operational efficiencies

(41)

Obtain the health & performance summary of specific Big Data technology

element

Visualize – your big data infrastructure from the top

Hadoop Dashboard

• Real time reflection of

topology & health

• Instant information on

availability of services e.g.,

HDFS, MapReduce

• Notifications on

performance & workload

Management Cockpit on

IOS and Android too

(42)

Big Data monitoring use cases

Ensure Hadoop jobs are

completed on optimal time

 Monitor available M/R slots,

slots used, completed and

waiting jobs

 Alert on high wait time or

failure of jobs

 Visualize map and reduce

slot usage trends Understand

the impact of failed jobs

 Remediate: Realign the

Hadoop cluster for optimal

performance

Ensure analytics reports are

executed on time on Vertica

 Monitor the resource queues

& rejection, lock and query

execution time

 Alert on lock, execution time,

long queue or resource

rejection

 Analyze real time & historical

data together, Check for

performance trends

 Remediate: Create resource

pool for high priority requests

Avoid any data loss on

Vertica cluster

 Monitor and alert on node

down, not K-Safe and critical

node state

 Visualize the cluster availably

and impact on your service

topology

 Report on capacity planning

& forecasting

 Remediate: Immediately

restore any failed /Shutdown

node.

(43)

Maintain the health of your dynamic infrastructure

Automated

End to end

Integrated

HP Solution: Big Data Monitoring Cockpit

Ensure Performance and

high availability for

big-data middleware

across network, storage,

compute resources

Fast, dynamic, and

cost-effective monitoring

Monitoring scale-up and

scale-out with the big-data

platform.

Quick and consistent

trouble response

integrated in existing 24x7

support operations.

(44)

Out of the box | Available today | More to come

OMi: Big Data Monitoring Cockpit

Hadoop

Distributed storage &

application processing

Vertica

Interactive real-time

analytics

Discovery

Data

Collection

Availability

monitoring

Performance

monitoring

Event

Correlation

Real time

graphs

Trend

Reporting

Infrastructure

System, OS, Virtual OS,

Cluster

Variety of supported

operating systems /

virtual operating systems

- CPU

- Memory

- Disk

- Network

- Resources

Monitor 1000s of

Hadoop nodes & clusters

through a single

dashboard

- MapReduce

- Job Statistics

- RW ops Throughput

- HDFS availability

Maintain the

infrastructure analyzing

your big data

- Query Performance

- Resource rejection

- Process status

- Node status

(45)

Reduce cost, Reduce risk, Increase confidence

HP Management Cockpit helps customers

Increase service availability

Vodafone reduced downtime by 77%, increased availability to

99.9%

Minimize business impact when issues happen

Florida Blue lowered the time to resolve to problems by 80%

Reduce cost

(46)

Call to action

• Free downloadable software

• White Papers

• Videos

• Recorded webinars

• Customer use cases

• Industry experts’ opinions

HP.COM/GO/OpsAnalytics

Download HP Operations

Manager I (OMi) and OMi

Management Packs Today!

Plus!

Download the new OMi White Paper

:

Intelligent IT Operations. Making IT smarter with

advanced event correlation and management

(47)

(48)

Thank you

• Complete the short survey and opt-in for

more information from HP.

www.hp.com