IoT Edge Processing. Evolution of edge computing analytics and long-term data retention. JEFF KIBLER VP Tech Services, Infobright

(1)

13 October 2015

AllSeen Alliance

1 IoT Edge Processing

JEFF KIBLER (@jrkibler)

VP Tech Services, Infobright

Evolution of edge computing analytics

and long-term data retention

(2)

1. IoT Premise and Challenges

2. Exposing Opportunities

3. Directions to Consider

4. Moving from Possible to Practical

5. Wrap-up

(3)

3 IoT Foundations

(4)

The Life of the End User

Athletics

Multi-billion dollar

industries where 1%

competitive edge

decides careers.

Infrastructure

Increasing reliance on

alternative energy,

permeable surfaces,

and environmental

metering.

Telemetry

Predicting and

improving health

outcomes.

(5)

13 October 2015

AllSeen Alliance

5 Premise

Leading Verticals

IoT Presents a Large Market Opportunity

Leading Challenges

• Data Security

• Infrastructure

• Privacy

• Governance

• Industrial Equipment

• Oil/Gas/Energy

• Automotive

• Retail / Restaurants

• Hospitals

• mHealth/teleHealth

• Infrastructure

(6)

Premise

IoT Solutions Today are Sexy, Self-Contained

Al

Cloud Based

Central Repository

Sensors

_Workflow

Rules/

Alerts,

Triggers,

Actions

Data:

NoSQL: Hadoop (Cloudera, Hortoworks, MapR), Cassanndra, MongoDB

Analytic: Sybase/IQ, HP Vertica, Amazon Redshift, Infobright, Pivitol

Standard Relational: Postgres, MySQL, Oracle, Sybase, Microsoft

Cloud: Amazon, Rackspace, Dimension

Data, Joyent, Cisco, EMC, IBM, Microsoft

Rules/Workflow: Apache Storm, Tibco Streambase,

Software AG Apama, Sybase Aleri, Various Coded in

(Java, Python, Ruby on Rails), TempoIQ

Closed Loop

Message-Response System

(7)

13 October 2015

AllSeen Alliance

7 Key Challenge

Added Complexity

Evolve with Simplicity

Centralized Volumes

• Gigabytes to Terabytes

• Terabytes to Petabytes

• Petabytes to Exabytes

• Data Exploitation Demands

• Edge Processing Demands

• Governance, ownership

(8)

Deliver an IoT Platform that

contemplates enormous

sophistication and complexity in a

delivery model that is intuitive,

(9)

9 IoT Foundations

(10)

Gaps Exposing the Opportunity

Data

Major Considerations

• How/where to leverage utility

value of data

Edge Processing

• Drivers behind and rationale of

edge processing both physical

and/or virtual

Architecture

• Meeting market requirements

over time; getting it right today

(11)

11 Many will overkill to address the

gaps. The result will be sophisticated

yet hardly elegant solutions.”

(12)

Viewpoints: Now and Future

Vendors and Users

Sample Industry Viewpoints Current Equipment Vendor

Drivers Current IoT User Drivers

Vendor Assumptions about the

Future User Assumptions about the Future

Industrial Equipment (Lutron Lighting / Glidden Paints)

Better product, higher margins, differentiation, stickiness

Higher uptime; easier servicing when needed; better results

Control of the silo, data, and devices. Customers will want the value add of accessing the data

Devices supplied by multiple vendors will work together and the data can be leveraged

Oil & Gas (FMC / Chevron)

Better products; safer products; proactive servicing

Safety; efficiency; visibility; uptime; compliance readiness

Gain product insight and control devices; value added services

Integrated IoT devices; holistic view from rig level up

Automotive (Ford / you)

Better product info; maintainability; increased margins; more competitive

Ease of use; comfort, safety; entertainment

Increasingly autonomous; changing models; compliance

Fully integrated experience, “car as device” including data; insurance; ownership

Retail & Restaurants (Viking Commercial / McDonald’s)

Tracking (Beacons); better equipment maintenance; higher uptime; customer stickiness

Higher yields per customer; better operational information; better uptime; greater sales

Greater level of integration required;

anonymization requirements

Leverage various IoT silos to create operational efficiencies and greater profits

Hospitals

(Lutron Lighting / Mercy Health St. Louis)

Higher uptime; greater efficiencies; enhanced supply chain

Less shrinkage; better compliance; greater visibility

Silo control with regulatory oversight; Integrated product suites

Exceptional level of integration of patient data and resources; operational efficiency

mHealth / Telehealth (New England BioLabs / McDonald’s)

“must have devices” for consumers; highly cost effective monitoring solutions

Health maintenance; physician accessibility; reduced costs; better outcomes

Lower cost delivery; shrinking footprint – becoming invisible; Lower energy; multi-point; integration

Greater exposure to data; integration with home systems; non-intrusive; lifestyle insights

Smart City Infrastructure

(Siemens / City of Chicago)

Specific silos

(lighting/rubbish/streets.

Increased efficiency and reduced cost of service delivery for various silos

Increasing footprint and product suite offerings; Mega vendor based service led engagements

Coordination and orchestration of holistic data; lower cost and better service delivery through analytics

(13)

13 October 2015

AllSeen Alliance

13 Viewpoints: Now and Future

Vendors and Users

Sample Industry Viewpoints Current Equipment Vendor

Drivers Current IoT User Drivers

Vendor Assumptions about the

Future User Assumptions about the Future

Industrial Equipment (Lutron Lighting / Glidden Paints)

Better product, higher margins, differentiation, stickiness

Higher uptime; easier servicing when needed; better results

Control of the silo, data, and devices. Customers will want the value add of accessing the data

Devices supplied by multiple vendors will work together and the data can be leveraged

Oil & Gas (FMC / Chevron)

Better products; safer products; proactive servicing

Safety; efficiency; visibility; uptime; compliance readiness

Gain product insight and control devices; value added services

Integrated IoT devices; holistic view from rig level up

Automotive (Ford / you)

Better product info; maintainability; increased margins; more competitive

Ease of use; comfort, safety; entertainment

Increasingly autonomous; changing models; compliance

Fully integrated experience, “car as device” including data; insurance; ownership

Retail & Restaurants (Viking Commercial / McDonald’s)

Tracking (Beacons); better equipment maintenance; higher uptime; customer stickiness

Higher yields per customer; better operational information; better uptime; greater sales

Greater level of integration required;

anonymization requirements

Leverage various IoT silos to create operational efficiencies and greater profits

Hospitals

(Lutron Lighting / Mercy Health St. Louis)

Higher uptime; greater efficiencies; enhanced supply chain

Less shrinkage; better compliance; greater visibility

Silo control with regulatory oversight; Integrated product suites

Exceptional level of integration of patient data and resources; operational efficiency

mHealth / Telehealth (New England BioLabs / McDonald’s)

“must have devices” for consumers; highly cost effective monitoring solutions

Health maintenance; physician accessibility; reduced costs; better outcomes

Lower cost delivery; shrinking footprint – becoming invisible; Lower energy; multi-point; integration

Greater exposure to data; integration with home systems; non-intrusive; lifestyle insights

Smart City Infrastructure

(Siemens / City of Chicago)

Specific silos

(lighting/rubbish/streets.

Increased efficiency and reduced cost of service delivery for various silos

Increasing footprint and product suite offerings; Mega vendor based service led engagements

Coordination and orchestration of holistic data; lower cost and better service delivery through analytics

Connected

Products

System of

Systems

(14)

Deliver an IoT Platform that

accommodates evolving user needs

with minimal user requirements.”

(15)

13 October 2015

AllSeen Alliance

15 Lens of a Vendor

Need

Considerations

• Product that performs and

adaptable

Data Characterization

• Framing view of product by data

Data Use

• Predict and Evolve Product

Constituencies

• Understand User Segmentation

Ownership

• Retain rights to Data

Stewardship

• Data Access by users

Management

• Controlling devices in the field

Drivers

• Decreased downtime, increased

utilization and visibility, Upsell

Outlook

• Integration into larger system of

systems

(16)

Lens of a User

Need

Considerations

• Operate efficiently to gain better

insight / make better decisions

Data Characterization

• Products and Services

Data Use

• Holistic understanding on data

breadth / avoid silos

Constituencies

• Organization or consumer

including various silo systems

Ownership

• Own the data

Stewardship

• Determine user permission

Management

• Product companies manage

assets

Drivers

• Cost savings, enhanced

outcomes, increased revenue

Outlook

(17)

17 IoT Foundations

(18)

IoT Direction

Pushing Suppliers for more Robust Analytic Stack

Al

Cloud Based

Central Repository

Sensors

_WorkflowRules/

Closed Loop

Message-Response System

Enterprise Apps:

ERP, CRM, and

other enterprise

apps

Possible Specialized Store

Alerts, Triggers,

Actions

Analytic Workbench: Operational, Investigative, Predictive Analytics and Machine Learning

(19)

13 October 2015

AllSeen Alliance

19 IoT Direction to the Edge

Increase in Edge Processing for filtering and increased capabilities

Cloud Based

Central Repository

Sensors

_WorkflowRules/

Closed Loop

Message-Response System

Enterprise Apps:

ERP, CRM, and

other enterprise

apps

Possible Specialized Store

Alerts, Triggers,

Actions

Edge Processor

• Apply rules and workflow against that data

• Take action as needed

• Filter and cleanse the data exhaust (increasing payload)

• Store local data for local use

• Enhance security

• Provide governance admin controls

Rules/

Workflow

(20)

Edge Processing Assumptions

• Limited or no human resources for maintaining the database or other system

capabilities at the edge – must be a hands off operation, with remote monitoring or

control only

• Hardware footprint will be limited

• Not all use cases apply – but many do

–

Factories

–

Retail/Restaurants

–

Homes (but with far less data)

–

Buildings

–

Many aspects of smart cities

–

Hospitals (but only marginally for personal health)

–

Cars (Edge on board)

–

Other transportation modalities (especially planes,, trains and ships)

(21)

13 October 2015

AllSeen Alliance

21 Architectural Considerations

Cloud-Based Central Repository Cloud-Based Central Repository Sensors Rules/ Workflow (& Filtering) Sensors

Closed Loop

Message-Response System

Edge Processor Sensors Sensors

External Data

Persisted Store

P

u

b

lish

Analytic Workbench: Operational, Investigative, Predictive ERP, CRM, etc.

Various Sensor Devices

First Receiver Cloud-Based Central Repository Rules/ Workflow Analytic Workbench Enterprise Apps Cloud-Based Central Repository Rules/ Workflow Analytic Workbench Enterprise Apps Sensors Sensors Sensors Sensors

Vendor Corporate (“Lutron Lighting”) One of multiple vendor silos

User Remote Site (“McDonald’s/South Boston”)

User Corporate (“McDonald’s Head Office”)

Government (“USDA”)

S

u

b

scr

ib

e

Cloud-Based Central Repository Rules/ Workflow Analytic Workbench Enterprise Apps

(22)

Local Back End Data Provisioning

Sensors

Rules/

Workflow

(& Filtering)

Sensors

Closed Loop

Message-Response System

Sensors

External Data

Persisted Store

Analytic

Workbench:

Operational

Investigative

Predictive

ERP, CRM, etc.

Various Sensor

Devices & Silos

First Receiver

Cloud-Based Central Repository Rules/ Workflow Analytic Workbench Enterprise Apps Cloud-Based Central Repository Rules/ Workflow Analytic Workbench Enterprise Apps Cloud-Based Central Repository Rules/ Workflow Analytic Workbench Enterprise Apps

Sensors

Vendor Corporate (“Lutron Lighting”)

McDonald’s

Vendor Corporate (“Honeywell HVAC”)

Vendor Corporate (“Bosch Appliances”)

(23)

23 Data beyond a certain scale becomes

impossible to accommodate and use

without vast infrastructure and

(24)

Small Edge Node Volumes Today

Most data volumes today and in the

near future are exceptionally low by

some standards (like Telco and

Networking)

The key will be to provide the

underpinnings to service the full

analytic stack and feed enterprise

applications

Hotel Example

Sensors Deployed: 100,000

Avg. Message Interval: 5 seconds

Exhaust Rate: 100

Avg. Message size: 3kb

Data Retention Period: 30 days

Required Message Flow Capacity: 2.16M Messages/Hr

Required Storage: 2.59 TB

Al

Cloud Based

Central Repository

Sensors

_Workflow

Rules/

Closed Loop

Message-Response System

Alerts,

Triggers,

Actions

Enterprise Apps: ERP, CRM, and other enterprise apps Possible Specialized Store

(25)

13 October 2015

AllSeen Alliance

25 Future Unknown Edge Node Volumes

The combination of many silos with greater

reach along with the augmentation with

external data will create much higher

volumes over time, especially in certain

user cases

The ability to practically accommodate

massive amounts of data in the future

will be a critical consideration of IoT

architectures

Sensors Rules/ Workflow (& Filtering)

Sensors

Closed Loop Message-Response System

Edge Processor Sensors Sensors External Data Persisted Store

P

u

b

lish

Analytic Workbench: Operational, Investigative, Predictive ERP, CRM, etc.

Various Sensor Devices First Receiver

Cloud-Based Central Repository Rules/

Workflow

Analytic Workbench

Enterprise Apps

Cloud-Based Central Repository Rules/

Workflow

Cloud-Based Central Repository Rules/ Workflow Sensors Sensors Sensors Sensors Vendor Corporate User Corporate

Third Party – as needed

S

u

b

scr

ib

e

Analytic Workbench Enterprise Apps Analytic Workbench Enterprise Apps

(26)

Opportunity for Providers and Users

Increased Data Focus

& Analytic Capabilities

Edge & Tier Processing

wherever appropriate

Publish &

Subscribe Architecture

Leveraging the

Utility Value of

IoT Data

(27)

27 IoT Foundations

(28)

Metadata Leveraged Architecture

Establish Metadata at the point

of ingestion

Provide comprehensive query

tools contemplating a variety of

needs

E n d p o in t D e v ic e s

1

st

_Receiver

Edge Processors

1

st

_Receiver

Edge Processors

1

st

_Receiver

Edge Processors

Mid-Tier

Edge Processors

Mid-Tier

Edge Processors

Includes

Infobright Store

integrated with

Hadoop for

enhancing

analysis of

machine data

(29)

13 October 2015

AllSeen Alliance

29 Metadata Leveraged Architecture

E n d p o in t D e v ic e s

1

st

_Receiver

Edge Processors

1

st

_Receiver

Edge Processors

1

st

_Receiver

Edge Processors

Mid-Tier

Edge Processors

Mid-Tier

Edge Processors

Includes

Infobright/Metadata

Store integrated

with Hadoop for

enhancing analysis

of machine data

Leverages Metadata throughout the architecture

Common Tool Sets, Minimal Administration

Affordable and Accessible

(30)