IoT Edge Processing. Evolution of edge computing analytics and long-term data retention. JEFF KIBLER VP Tech Services, Infobright

Full text

(1)

13 October 2015

AllSeen Alliance

1

IoT Edge Processing

JEFF KIBLER (@jrkibler)

VP Tech Services, Infobright

Evolution of edge computing analytics

and long-term data retention

(2)

1.

IoT Premise and Challenges

2.

Exposing Opportunities

3.

Directions to Consider

4.

Moving from Possible to Practical

5.

Wrap-up

(3)

3

IoT Foundations

(4)

The Life of the End User

Athletics

Multi-billion dollar

industries where 1%

competitive edge

decides careers.

Infrastructure

Increasing reliance on

alternative energy,

permeable surfaces,

and environmental

metering.

Telemetry

Predicting and

improving health

outcomes.

(5)

13 October 2015

AllSeen Alliance

5

Premise

Leading Verticals

IoT Presents a Large Market Opportunity

Leading Challenges

Data Security

Infrastructure

Privacy

Governance

Industrial Equipment

Oil/Gas/Energy

Automotive

Retail / Restaurants

Hospitals

mHealth/teleHealth

Infrastructure

(6)

Premise

IoT Solutions Today are Sexy, Self-Contained

Al

Cloud Based

Central Repository

Sensors

Workflow

Rules/

Alerts,

Triggers,

Actions

Data:

NoSQL: Hadoop (Cloudera, Hortoworks, MapR), Cassanndra, MongoDB

Analytic: Sybase/IQ, HP Vertica, Amazon Redshift, Infobright, Pivitol

Standard Relational: Postgres, MySQL, Oracle, Sybase, Microsoft

Cloud: Amazon, Rackspace, Dimension

Data, Joyent, Cisco, EMC, IBM, Microsoft

Rules/Workflow: Apache Storm, Tibco Streambase,

Software AG Apama, Sybase Aleri, Various Coded in

(Java, Python, Ruby on Rails), TempoIQ

Closed Loop

Message-Response System

(7)

13 October 2015

AllSeen Alliance

7

Key Challenge

Added Complexity

Evolve with Simplicity

Centralized Volumes

Gigabytes to Terabytes

Terabytes to Petabytes

Petabytes to Exabytes

Data Exploitation Demands

Edge Processing Demands

Governance, ownership

(8)

Deliver an IoT Platform that

contemplates enormous

sophistication and complexity in a

delivery model that is intuitive,

(9)

9

IoT Foundations

(10)

Gaps Exposing the Opportunity

Data

Major Considerations

How/where to leverage utility

value of data

Edge Processing

Drivers behind and rationale of

edge processing both physical

and/or virtual

Architecture

Meeting market requirements

over time; getting it right today

(11)

11

Many will overkill to address the

gaps. The result will be sophisticated

yet hardly elegant solutions.”

(12)

Viewpoints: Now and Future

Vendors and Users

Sample Industry Viewpoints Current Equipment Vendor

Drivers Current IoT User Drivers

Vendor Assumptions about the

Future User Assumptions about the Future

Industrial Equipment (Lutron Lighting / Glidden Paints)

Better product, higher margins, differentiation, stickiness

Higher uptime; easier servicing when needed; better results

Control of the silo, data, and devices. Customers will want the value add of accessing the data

Devices supplied by multiple vendors will work together and the data can be leveraged

Oil & Gas (FMC / Chevron)

Better products; safer products; proactive servicing

Safety; efficiency; visibility; uptime; compliance readiness

Gain product insight and control devices; value added services

Integrated IoT devices; holistic view from rig level up

Automotive (Ford / you)

Better product info; maintainability; increased margins; more competitive

Ease of use; comfort, safety; entertainment

Increasingly autonomous; changing models; compliance

Fully integrated experience, “car as device” including data; insurance; ownership

Retail & Restaurants (Viking Commercial / McDonald’s)

Tracking (Beacons); better equipment maintenance; higher uptime; customer stickiness

Higher yields per customer; better operational information; better uptime; greater sales

Greater level of integration required;

anonymization requirements

Leverage various IoT silos to create operational efficiencies and greater profits

Hospitals

(Lutron Lighting / Mercy Health St. Louis)

Higher uptime; greater efficiencies; enhanced supply chain

Less shrinkage; better compliance; greater visibility

Silo control with regulatory oversight; Integrated product suites

Exceptional level of integration of patient data and resources; operational efficiency

mHealth / Telehealth (New England BioLabs / McDonald’s)

“must have devices” for consumers; highly cost effective monitoring solutions

Health maintenance; physician accessibility; reduced costs; better outcomes

Lower cost delivery; shrinking footprint – becoming invisible; Lower energy; multi-point; integration

Greater exposure to data; integration with home systems; non-intrusive; lifestyle insights

Smart City Infrastructure

(Siemens / City of Chicago)

Specific silos

(lighting/rubbish/streets.

Increased efficiency and reduced cost of service delivery for various silos

Increasing footprint and product suite offerings; Mega vendor based service led engagements

Coordination and orchestration of holistic data; lower cost and better service delivery through analytics

(13)

13 October 2015

AllSeen Alliance

13

Viewpoints: Now and Future

Vendors and Users

Sample Industry Viewpoints Current Equipment Vendor

Drivers Current IoT User Drivers

Vendor Assumptions about the

Future User Assumptions about the Future

Industrial Equipment (Lutron Lighting / Glidden Paints)

Better product, higher margins, differentiation, stickiness

Higher uptime; easier servicing when needed; better results

Control of the silo, data, and devices. Customers will want the value add of accessing the data

Devices supplied by multiple vendors will work together and the data can be leveraged

Oil & Gas (FMC / Chevron)

Better products; safer products; proactive servicing

Safety; efficiency; visibility; uptime; compliance readiness

Gain product insight and control devices; value added services

Integrated IoT devices; holistic view from rig level up

Automotive (Ford / you)

Better product info; maintainability; increased margins; more competitive

Ease of use; comfort, safety; entertainment

Increasingly autonomous; changing models; compliance

Fully integrated experience, “car as device” including data; insurance; ownership

Retail & Restaurants (Viking Commercial / McDonald’s)

Tracking (Beacons); better equipment maintenance; higher uptime; customer stickiness

Higher yields per customer; better operational information; better uptime; greater sales

Greater level of integration required;

anonymization requirements

Leverage various IoT silos to create operational efficiencies and greater profits

Hospitals

(Lutron Lighting / Mercy Health St. Louis)

Higher uptime; greater efficiencies; enhanced supply chain

Less shrinkage; better compliance; greater visibility

Silo control with regulatory oversight; Integrated product suites

Exceptional level of integration of patient data and resources; operational efficiency

mHealth / Telehealth (New England BioLabs / McDonald’s)

“must have devices” for consumers; highly cost effective monitoring solutions

Health maintenance; physician accessibility; reduced costs; better outcomes

Lower cost delivery; shrinking footprint – becoming invisible; Lower energy; multi-point; integration

Greater exposure to data; integration with home systems; non-intrusive; lifestyle insights

Smart City Infrastructure

(Siemens / City of Chicago)

Specific silos

(lighting/rubbish/streets.

Increased efficiency and reduced cost of service delivery for various silos

Increasing footprint and product suite offerings; Mega vendor based service led engagements

Coordination and orchestration of holistic data; lower cost and better service delivery through analytics

Connected

Products

System of

Systems

(14)

Deliver an IoT Platform that

accommodates evolving user needs

with minimal user requirements.”

(15)

13 October 2015

AllSeen Alliance

15

Lens of a Vendor

Need

Considerations

Product that performs and

adaptable

Data Characterization

Framing view of product by data

Data Use

Predict and Evolve Product

Constituencies

Understand User Segmentation

Ownership

Retain rights to Data

Stewardship

Data Access by users

Management

Controlling devices in the field

Drivers

Decreased downtime, increased

utilization and visibility, Upsell

Outlook

Integration into larger system of

systems

(16)

Lens of a User

Need

Considerations

Operate efficiently to gain better

insight / make better decisions

Data Characterization

Products and Services

Data Use

Holistic understanding on data

breadth / avoid silos

Constituencies

Organization or consumer

including various silo systems

Ownership

Own the data

Stewardship

Determine user permission

Management

Product companies manage

assets

Drivers

Cost savings, enhanced

outcomes, increased revenue

Outlook

(17)

17

IoT Foundations

(18)

IoT Direction

Pushing Suppliers for more Robust Analytic Stack

Al

Cloud Based

Central Repository

Sensors

WorkflowRules/

Closed Loop

Message-Response System

Enterprise Apps:

ERP, CRM, and

other enterprise

apps

Possible Specialized Store

Alerts, Triggers,

Actions

Analytic Workbench: Operational, Investigative, Predictive Analytics and Machine Learning

(19)

13 October 2015

AllSeen Alliance

19

IoT Direction to the Edge

Increase in Edge Processing for filtering and increased capabilities

Cloud Based

Central Repository

Sensors

WorkflowRules/

Closed Loop

Message-Response System

Enterprise Apps:

ERP, CRM, and

other enterprise

apps

Possible Specialized Store

Alerts, Triggers,

Actions

Analytic Workbench: Operational, Investigative, Predictive Analytics and Machine Learning

Edge Processor

Apply rules and workflow against that data

Take action as needed

Filter and cleanse the data exhaust (increasing payload)

Store local data for local use

Enhance security

Provide governance admin controls

Rules/

Workflow

(20)

Edge Processing Assumptions

Limited or no human resources for maintaining the database or other system

capabilities at the edge – must be a hands off operation, with remote monitoring or

control only

Hardware footprint will be limited

Not all use cases apply – but many do

Factories

Retail/Restaurants

Homes (but with far less data)

Buildings

Many aspects of smart cities

Hospitals (but only marginally for personal health)

Cars (Edge on board)

Other transportation modalities (especially planes,, trains and ships)

(21)

13 October 2015

AllSeen Alliance

21

Architectural Considerations

Cloud-Based Central Repository Cloud-Based Central Repository Sensors Rules/ Workflow (& Filtering) Sensors

Closed Loop

Message-Response System

Edge Processor Sensors Sensors

External Data

Persisted Store

P

u

b

lish

Analytic Workbench: Operational, Investigative, Predictive ERP, CRM, etc.

Various Sensor Devices

First Receiver Cloud-Based Central Repository Rules/ Workflow Analytic Workbench Enterprise Apps Cloud-Based Central Repository Rules/ Workflow Analytic Workbench Enterprise Apps Sensors Sensors Sensors Sensors

Vendor Corporate (“Lutron Lighting”) One of multiple vendor silos

User Remote Site (“McDonald’s/South Boston”)

User Corporate (“McDonald’s Head Office”)

Government (“USDA”)

S

u

b

scr

ib

e

Cloud-Based Central Repository Rules/ Workflow Analytic Workbench Enterprise Apps

(22)

Local Back End Data Provisioning

Sensors

Rules/

Workflow

(& Filtering)

Sensors

Closed Loop

Message-Response System

Sensors

Sensors

External Data

Persisted Store

Analytic

Workbench:

Operational

Investigative

Predictive

ERP, CRM, etc.

Various Sensor

Devices & Silos

First Receiver

Cloud-Based Central Repository Rules/ Workflow Analytic Workbench Enterprise Apps Cloud-Based Central Repository Rules/ Workflow Analytic Workbench Enterprise Apps Cloud-Based Central Repository Rules/ Workflow Analytic Workbench Enterprise Apps

Sensors

Sensors

Sensors

Sensors

Vendor Corporate (“Lutron Lighting”)

McDonald’s

Vendor Corporate (“Honeywell HVAC”)

Vendor Corporate (“Bosch Appliances”)

(23)

23

Data beyond a certain scale becomes

impossible to accommodate and use

without vast infrastructure and

(24)

Small Edge Node Volumes Today

Most data volumes today and in the

near future are exceptionally low by

some standards (like Telco and

Networking)

The key will be to provide the

underpinnings to service the full

analytic stack and feed enterprise

applications

Hotel Example

Sensors Deployed: 100,000

Avg. Message Interval: 5 seconds

Exhaust Rate: 100

Avg. Message size: 3kb

Data Retention Period: 30 days

Required Message Flow Capacity: 2.16M Messages/Hr

Required Storage: 2.59 TB

Al

Cloud Based

Central Repository

Sensors

Workflow

Rules/

Closed Loop

Message-Response System

Alerts,

Triggers,

Actions

Analytic Workbench: Operational, Investigative, Predictive Analytics and Machine Learning

Enterprise Apps: ERP, CRM, and other enterprise apps Possible Specialized Store

(25)

13 October 2015

AllSeen Alliance

25

Future Unknown Edge Node Volumes

The combination of many silos with greater

reach along with the augmentation with

external data will create much higher

volumes over time, especially in certain

user cases

The ability to practically accommodate

massive amounts of data in the future

will be a critical consideration of IoT

architectures

Sensors Rules/ Workflow (& Filtering)

Sensors

Closed Loop Message-Response System

Edge Processor Sensors Sensors External Data Persisted Store

P

u

b

lish

Analytic Workbench: Operational, Investigative, Predictive ERP, CRM, etc.

Various Sensor Devices First Receiver

Cloud-Based Central Repository Rules/

Workflow

Analytic Workbench

Enterprise Apps

Cloud-Based Central Repository Rules/

Workflow

Cloud-Based Central Repository Rules/ Workflow Sensors Sensors Sensors Sensors Vendor Corporate User Corporate

Third Party – as needed

S

u

b

scr

ib

e

Analytic Workbench Enterprise Apps Analytic Workbench Enterprise Apps

(26)

Opportunity for Providers and Users

Increased Data Focus

& Analytic Capabilities

Edge & Tier Processing

wherever appropriate

Publish &

Subscribe Architecture

Leveraging the

Utility Value of

IoT Data

(27)

27

IoT Foundations

(28)

Metadata Leveraged Architecture

Establish Metadata at the point

of ingestion

Provide comprehensive query

tools contemplating a variety of

needs

E n d p o in t D e v ic e s

1

st

Receiver

Edge Processors

1

st

Receiver

Edge Processors

1

st

Receiver

Edge Processors

Mid-Tier

Edge Processors

Mid-Tier

Edge Processors

Includes

Infobright Store

integrated with

Hadoop for

enhancing

analysis of

machine data

(29)

13 October 2015

AllSeen Alliance

29

Metadata Leveraged Architecture

E n d p o in t D e v ic e s

1

st

Receiver

Edge Processors

1

st

Receiver

Edge Processors

1

st

Receiver

Edge Processors

Mid-Tier

Edge Processors

Mid-Tier

Edge Processors

Includes

Infobright/Metadata

Store integrated

with Hadoop for

enhancing analysis

of machine data

Leverages Metadata throughout the architecture

Common Tool Sets, Minimal Administration

Affordable and Accessible

(30)

Gap: Leveraging Data and Analytics

Lack of data leverage and more robust analytic stack will become an increasing

impediment

Gap: Edge Processing

Edge processing is cloud based filtering and workflow for exhaust

Gap: Publish/Subscribe Model

Basic monolithic cloud architectures

Opportunity: Data Cleansed, Enriched, and Published

Analytic stack can be established to provide operational, investigative, predictive,

and machine learning.

Opportunity: Edge Processing / First Receiver

Extensible version of workflow and data cleansing for edge deployments

Opportunity: Event-driven Architecture

(31)

13 October 2015

AllSeen Alliance

31

Thank you

Follow us on

For more information on AllSeen Alliance, visit us at:

allseenalliance.org & allseenalliance.org/news/blogs

Figure

Updating...

References

Updating...