SOA Connectivity & Integration Real World Examples

(1)

SOA Connectivity & Integration …

Real World Examples

Rachel Reinitz

Distinguished Engineer

[email protected]

(2)

Agenda

 

My Disclaimer

– 

I’ve changed customer details to hide their identities.

 

Speedy Expansion of Hotel Loyalty Program

 

Improving Museum Relationships thru Adoption of SalesForce.com

 

Event Driven and Location Based Marketing

 

Business Process Outsourcing (BPO) Company Expands to Mid-sized

Customers

(3)

Modern Win-Win Business

Partnership

Company A

Company B

Characteristics (different from classic B2B)

Fast real-time

Over the Internet (vs VPN)

XML/modern API

(4)

Speedy Expansion of Hotel

Loyalty Program

(5)

Architecture

Retailer

Public

APIs

DB

Loyalty

Program

Loyalty

Payment

Processing

ESB/XML Appliance

REST/HTTPS • Security

• Calling stored procedures • Converting to REST format • Logging for monitoring • Fast, fast, fast

• Key requirements

• Easy to sign up for program

• Secure exchange of customer information

• 50 ms response time and high scale

New stored procedures

(6)

From start of project to ‘go live’

less than 3 months

Technology

  ESB appliance addressing need for

low latency services with high

availability

  Use of standard configurations

delivered quick, consistent results

– Developed security & logging

standard infrastructure for ESB

– Developed ESB patterns to be

supported

  Stored procedures provided quick

retrieval of the data

  Included custom service monitoring

  On-line retailer had a lot of

experience with exposing APIs to

partners

Organization/development

practices

  High visibility & strong executive

support

  Had built skills on ESB for 6 months

before project start thru education

and expert mentoring

  Developed standardized operational

procedures with scripts

(7)

Increase agility and lower cost

through SaaS Adoption

Old System

Software as a Service

(8)

Improving Museum

Relationships thru Adoption of

SalesForce.com

8

SaaS Customer

Relationship Mgr

(9)

Key Decisions

We don’t want to operate infrastructure!

Move all applications to public cloud over time.

Try out moving to the cloud

Moved from Outlook to Gmail

We need a new, modern CRM system!

Let’s adopt SalesForce.com

Hmm..But we can’t move all at once, as lots of other legacy application use AS/

400 data

We need to integrate SalesForce.com and AS/400 and migrate data. Should

we build or buy integration?

Let’s buy a cloud integration product running in the public cloud.

Let’s evaluate leading cloud integration vendors.

We choose IBM Cast Iron Live!!!

We don’t have skills on Cloud integration/Cast Iron - how do we build

our integrations?

Let’s bring in Cast Iron experts; use local integration company; hire and build

(10)

Technical Solution

  6 orchestrations developed by IBM from for SalesForce to AS/400 DB

  4 orchestrations developed by a local partner learning Cloud Integration/

WebSphere Cast Iron Live) for AS/400 DB to SalesForce

  Leveraged the extensive connector/API support for SF and DBs

  Orchestrations run on scheduled basis, ex. once an hour

  Data migration done with scripts, but could have been done faster with

orchestrations

SalesForce

.

com

Cloud to

On-premise

Integration as a

Service

• AS/400 Secure Connector

Museum

SFDC APIs Over HTTPS SQL

(11)

So What Happened?

3 month project

• When you present your membership card, the cashier is using SalesForce custom screens instead of AS/400 bluescreen.

• New processes enabled - supporting new member targets and retention rates

• End users happy – no more Excel spreadsheets • Sponsors happy

• Budget and schedule meet (the manager wisely included buffer)

• Development & testing effort for the 6 orchestration – 12 elapsed weeks. But could have been done cheaper and if:

• data migration was done using Cast Iron instead of scripts • SalesForce configuration more stable

• personnel being mentored had been stable • could have been done in 5-6 weeks

• Some rework is being done on the local integrator’s Cast Iron integrations due to improve performance and fix suboptimal design choices

• Work is going on to expand use of SalesForce and build new integrations • Future: Decrease annual spend to support the system infrastructure.

Lessons learned from Museum IT Team:

• A cloud strategy frees up people and resources.

• Invest in applications which allow you to do more with les • Partner with companies whose solutions make sense for your business need

• It’s all about the Data!

• Involve product experts to “get it right the first time”; adopt a “learn to fish” strategy

• Standard best practices still apply (e.g., source code control, change management, structured QA process)

(12)

Leveraging Complex Events

Processing and Analytics with

Mobile to Deliver New Capability

!"!"##$%&'$()*+)*,-.)/ ,00)1/-2 )*34*2 0,242 )++)*-1/.-.42 $,00)1/-$34-,.52$6,271+

8(!

9$*,-4$5.6.-./: 9$*42+)/24$0,07./: 9$;65$!$<2)/ 9$1*5$*4=*.-./: 9$,/,5>-.02 $ ?'@ ABCD

E*)+)2,5$9$

,+.

:44

$F$

(GBH$%ICD

$EC(

',271+$3,-,$2)1*042$./-)$GE%2 E*)J.34$GE%$0)/-*)5543K$ =.-7$./2.:7-$,/3$20,54 %&'$()/L.34/-.,5

Engage

Phone

User

(13)

Event Driven and Location

Based Marketing

13 !"!"##$%&'$()*+)*,-.)/ ,00)1/-2 )*34*2 0,242 )++)*-1/.-.42 $,00)1/-$34-,.52$6,271+

8(!

9$*,-4$5.6.-./: 9$*42+)/24$0,07./: 9$;65$!$<2)/ 9$1*5$*4=*.-./: 9$,/,5>-.02 $ ?'@ ABCD

E*)+)2,5$9$

,+.

:44

$F$

(GBH$%ICD

$EC(

',271+$3,-,$2)1*042$./-)$GE%2 E*)J.34$GE%$0)/-*)5543K$

=.-7$./2.:7-$,/3$20,54

%&'$()/L.34/-.,5

My Telco Retailers design offers

I sign up for offers

1

2

3

(14)

Interaction Flow

Send SMS or app Message ‘I’m shopping’

Customer

Use the coupon barcode Or ID when purchasing

item

Receives coupon for RetailerX offer via SMS

Filter events for customers

signed up & type of event Determine location based on GPS

Process coupon rules – For CustomerY at LocationZ determine if eligible for any coupons; if a coupon should be sent; and if so,

which one based on many criteria

Send coupon to customer

Track what coupons are used as input for offering rules; customer profile; and business monitoring

Signs up to get coupons From RetailerX

Event Processing & Decision System

Customer Registration WebSite Campaign Mgnt Website

Coupon is applied by retailer system (like other coupons)

1b 2 3 4 6 5 8 a 7 8b

Retailer Signs-up & Configures Coupon offers

(15)

Event Processing Architecture

Event Processing & Decision System

Messaging

• Transport SMS events

Complex Event Processing & Business Rules

• Apply rules controlling frequency of offers (time based)

• Apply rules wrt to location proximity

• Apply rules about order/priority coupons across merchants

• Apply rules to select amongst competing coupons

• …..

ESB and Basic Event Processing

• Event filtering (shopping, coupon use) • Get location through GPS

• Get nearby enrolled stores through DB spatial • Apply simple coupon rules through DB queries • Update DBs

• Compose coupon messages • Convert and transport for SMS

High

performance ESB/XML Security gateway

• Filter for customers signed up for coupons • Data validation • Security Database with Spatial Extender • Offering DB • Store locations • Coupon History • ….

(16)

So What Happened?

Results:

• Business lead initiative by innovative business analysis

• With IBM developed a pilot version with one retailer

• Retailer happy with results

• Demoed pilot live to executives -> excitement

• Being expanded significantly,

incrementally over last year and going strong

What worked well:

• Partnership with business

• Partnership with IBM development labs and services

• Agile development with updates every 6-8 weeks

• Overall architecture works well

Challenges:

• Overconfident telco architects who designed the flows -> bad initial performance

• Different scalability/performance characteristics of existing systems events flow thru from the main event processing

• Incremental adding of new capability, particularly variety of rules, without performance testing

• Lack of investment in properly training telco developers lead to dependency on IBM Software Services

• Questionable requirements around meta-data driven, dynamic changes led to custom complex event processing code

• Custom UI for merchants needed Future:

• Expanding to more retailers

• Allow more variety of offering rules • Add analytics to customize offerings for individual

(17)

Lowering Operating Expense

thru Business Process

Outsourcing (BPO)

(18)

Business Process Outsourcing

(BPO) Company Expands to

Mid-sized Customers

18

Outsourced

Business

Processes

Client-specific

information

(SAP, Oracle, etc)

Pre-canned “plugs”

Connecting Client Info

with Business

Processes

Custom “plugs”

Connecting Client Info

with Business

Processes

BPO

Provider

Large-ish

Client

Mid-sized

Client

Client-specific

information

(19)

What does it take for a mid-sized

customer to use Tax Processing

Services from a BPO?

BPO Company

ERP Custom code to

•  extract data from backend

•  apply transformation & business rules

•  convert into provider API format

•  connect & transmit to provider Tax Processing

Service

Customer

  BPO exposes Service APIs for sending payroll and tax data

  Complex, frequently changing business rules must be applied to backend Enterprise Resource Planning (ERP) information to summarize data for tax processing.

  Integration cost for mid-size customers is a significant inhibitor to adoption of the BPO tax services

  BPO needs to provide a pre-packaged, lower cost, faster, simpler integration option for mid-size customers and partnered with us to provide it:

–  Key to lowering integration cost is having repeatable, easily customized solution –  delivering a virtual appliance solution customized for the specific ERP and customer

configuration which installed at the customer site

SOAP/ HTTPS Secure FTP

(20)

BPO Solution – Component

Architecture

Orchestration with Mappings To Provider

Business Rules Engine

Canonical Database for Domain Management UI

BPO

Customer Backend System Mappings for specific 3rd_Party to Canonical Mappings for Customer Customization to Canonical Tax Service

BPO Integration Virtual Appliance

Tax Service Focused Custom for customer Specific to the provider

Customer Premises

Solution components 1 2b Orchestration 2 a 3 4 Domain Business Rules 6 5 7 Management UI Operations UI

(21)

BPO Integration Lessons

Learned

 Smaller customers are very cost sensitive

– Need to ‘right size’ hardware specification

– Lower cost thru Xen and newer VMWare pricing

– Support a cloud deployment option with minimal customer

install

 Having well defined Service API is critical

 Customization of solution vary significantly across domains

– Payroll system structure vary a lot – even across

implementations of same ERP

– Transformation and business rules may require deep domain

expertise

– Analysis of set of customers backend systems is critical for

determine commonality of 3

rd

_{party systems}

– Need to understand cost to develop the custom part of the

solution

 Maintenance

– Mid-sized customers have issues with data integrity

• Added ‘balance checkers’ as part of the BPO management

solution

(22)

Raw data

Secure Connector

Future Cloud Deployment Option

for BPO Integration Solution

BPO

Customer

IBM BPO Integration Solution (Orchestrations, DB, Rules, Management UI) Provider Service

Reduced data

IBM Cloud

• IBM Cloud Solution with very small installed program at customer site which then

connects to the backend system

• Transformations and rules still customized for customer configuration

• Low cost, least intrusive option for a customer

• Requires 2 network hops – for large amounts of data incur significant latency and/or

require big network pipe from customer

(23)

Raw data Secure _Connector

Deployment options – Provider

Cloud

Provider

Customer

BPO Integration Solution

(Orchestrations, DB, Rules )

Provider Service

Reduced data

• Eliminates data security issues since customer has to send data to the provider in any

case

• Maybe sending large amount of raw data to provider

• Requires BPO to manage cloud environment

(24)

Extreme Innovation - Solve the

World’s Hunger Problem

  Key Facts

–  Every 3.6 seconds someone dies of hunger

–  1 in 12 people worldwide is malnourished

–  To satisfy the world's sanitation and food requirements would cost only US$13 billion- what the people of the United States and the

European Union spend on perfume each year.

Root Cause and Scenario Connect

- There is enough number of people with the means to satisfy the Hunger problem.

– - They don't or are unable to channelize their help due to the following reasons:

– Lack of Transparency (People don't know where their charity money was spent).

– Lack of ease to donate. (Lack of proper gateway).

– Difficult to track if the charity money was actually spent for the intended purpose. (was not miss-utilized).

– People don’t usually carry cash in their pockets all the time.

– - Scenario: You may give $5 to a beggar hoping that he will buy some food from it... but in reality

he might just purchase alcohol/drugs with that money. Thus, the intention was not met on the contrary the help money was misused.

  Reason for Choice

–  Universal and Affects masses across continents

–  Still unsolved

–  We believe that we can make use of the current technology in Hybrid Cloud

Integration to deal with this problem of Hunger and also bring more

transparency in the way people do Charity.

(25)

SOA Connectivity & Integration Real World Examples