Deploying SOA using IBM s Enterprise Integration Messaging Specification (EIMS)

E-Business On Demand

Deploying SOA using IBM’s Enterprise

Integration Messaging Specification (EIMS)

Agenda

Why deploy a Service Oriented Architecture (SOA)?

Why are messaging standards important?

What Exactly is EIMS?

Getting There – The EIMS Development Process

e-business On Demand

An enterprise whose business processes—integrated

end-to-end across the company and with key partners,

suppliers and customers—can respond with speed to

any customer demand, market opportunity or external threat.

—Sam Palmisano IBM Chief Executive Officer

“

”

Process

People

SOA is transforming IBM to enable faster solution development and on demand adaptability. Business Units are able to create new services quickly, and to incrementally transform legacy and packaged application functions into “Services”. Assets (including messaging data) is architected for reuse to support the deployment of service based applications across the enterprise.

Extracted and transformed segments of legacy/packaged applications

Business analysts compose ‘Plug & Play’ modular business processes, with pre-associated SOA components, to rapidly implement business rule driven, easily adaptable, operational workflows.

Modular Process IT Component IT Component Web Services Compose Deploy

IBM On Demand Development Environment Tool Suite (Rational & Websphere)

Workflow

Monolithic processes and applications can’t be reassembled or reused. Ad hoc integration connections are difficult to change and maintain. Lack of standards limits meaningful interoperability. Rigidity of architecture makes small improvements impossible to cost justify.

Why SOA? Companies are unable to keep pace with market demand for new business capabilities. Processes are not dynamically adaptable. Architecture is a choke point for business innovation.

What are the Architectural Objectives of the SOA EIMS Messaging Architecture?

Loose coupling - Service application interfaces are specified and managed in a manner that

will allow service functions to be loosely coupled, location transparent and protocol independent;

Application Re-engineering - Will support the evolution of IBM’s internal application landscape from one that encompasses a large number of proprietary solutions (4800+), to a more

flexible and resilient architecture, with significantly less redundancy;

Common business data representation - Existing applications/components can be leveraged

to provide their services using a common business data representation. Existing functions can be “represented” using common data vocabularies, so that the services provided by these functions can be available to the enterprise in a consistent way, regardless of its proprietary interface;

Integration simplification - SOA applications are integrated at the service specification point, not at the application interface. This allows application complexities to be isolated from the services interface, and minimizes the impacts of back-end interface and application changes;

Responsive to change - Provides an environment whereby business services workflow and

business process re-orchestration can be accomplished easily, without the need to go through costly and time consuming software development cycles;

SOA & EIMS Concepts..

SOA is more than a technology. It is a management and design philosophy that can be used to bridge the divide that often exists between Business Owners and Technology Developers.

- Application integrators will re-engineer the Enterprise by designing application interfaces that provide standardized and re-usable “Services”, rather than application specific interfaces.

- Business Process Owners will define and orchestrate granular “Service Components” for their Business Domains that can be used to optimize their businesses.

The Enterprise Integration Messaging Specification (EIMS): OAGIS based integration

architecture that has been “extended” to meet internal application-to-application integration

and SOA requirements. An effective SOA messaging architecture will utilize re-usable and

standardized message structures and data vocabularies.

SOA solutions are typically defined using UML. SOA message payloads are defined using EIMS processes. The EIMS development process defines tools, governance and design and techniques that facilitate rapid solution development.

It is fairly simple to generate web services using programming tools without being concerned about data. However, this approach makes it difficult to re-use common data structures and

End State Vision

Distributors Manufacturers Customers Trading Partner External Protocols Products & Solutions: Personal Computers Servers Software Storage Services Edge Of Enterprise Ariba TRADING PARTNERS CommerceOne SAP RosettaNet Oracle Web Browser EDI

Service based messaging (using EIMS) enables internal and B2B solutions

Internet

EIMS

Internal Applications IBM Internal Message Hubs Fulfillment Manufacturing Services Web Services

EIMS

(A2A)

. . . .

Why are messaging standards important?

Problems: Many legacy applications use proprietary interface data definitions. Why is this a problem?

- Application Integration is very expensive Each new interface is a custom solution. - Infrastructures are costly Many point-to-point interfaces exist between IBM’s applications.

- Business Processes are difficult to modify Every process change requires interface

customization.

- Application services are difficult to re-use Messages are defined using application specific data semantics.

Solution:

Develop an enterprise messaging architecture and standardized message vocabulary that can be used across the corporation for application integration, and deployment of IBM’s Service Oriented Architecture (SOA).

Benefits of SOA and a Common Messaging Specification

Legacy: Many Point-to-Point Interfaces (Required when Interface Data is unique)

Service Oriented Topology

(Common vocabularies + Reusable Services)

SOA and Common Messaging Specification: 2 Reusable Messages

(16 Message Interfaces/Implementations) Point-to-Point Architecture:

Potentially, 56 Unique Messages (112 Message Interfaces/Implementations) 2*N(N-1) Message Interfaces A B C D H G F E 2 Common Messages 2*N Message Interfaces A E F G H B C D

The Big Picture

EIMS Vocabulary

EIMS – Typical B2B Examples

E IM S C re a te /E x tr_a c t S y_s te m

ORDER

EIMS-BOD

(Order PBI and variants) EIM S Cre ate /Ex tra ct Sy ste m E IM S C re a te /E x tr a c t S y s te m Multiple Trading partners, Multiple formats: Ariba/cXML Rosetta Net SAP OAGi C1/xCBL Custom XML Web Services EDI Application(s) that map to/from the EIMS Web Interfaces: eSites Partners Public EIMS-BOD – Canonical Business Messaging Object (Generic Business Object in WBI-ICS) Application(s) that map to/from the EIMS Application(s) that map to/from the EIMS IBM’s Back End Systems

Process Services Community Integration Services Enterprise Applications

EIMS Supports the Deployment of the WebSphere

Business Integration Reference Architecture

Enterprise Service Bus Application

Services

Development Tools, Services Model, Design, Development, Test Tools

Monitoring Services

Data Access Services

Enterprise Data

User Interaction

Services

Common Runtime Infrastructure

Application Access Services

Information Services Delivery Services Experience Services Resource Services J2EE Extensions J2EE Container Svcs

J2EE Base Services

Business Trans Svcs Process State Mgmt Staff Services Choreography Svcs _{Community Mgmt} Document Mgmt Protocol Services

IT Monitoring Process Monitoring

Replication Services

Transformation Svcs Search Services Federation Services

Transport Services Mediation Services Event Services

On-ramp Services Event Detection Services

On-ramp Services EIMS Data Dictionary Is used for Message Transformation EIMS Data vocabulary

is used for ESB Mediation

EIMS Messages Support BPEL

Enterprise Integration Messaging Specification – in a Nutshell

The Enterprise Integration Messaging Specification (EIMS) is the message payload

used for A2A within IBM. It expands OAGIS R8 (Open Applications Group

Integration Specification) and possesses the following characteristics:

–

Standardized Messaging Interaction Definition

–

Best suited for Enterprise Web Services

–

Standardized Messaging Structure facilitates re-use

–

Guiding Principles and Methodology

–

Canonical Model and Precise Business Interaction (PBI) schemas for transactions

–

Common Building Blocks and Corporate Assets

–

Development Process: Tools, Work Products, Deliverables and Work Flow

–

Governance and Related Standards

–

EIMS Development Process

–

Business Transformation Management System (BTMS)

–

Web Standards and Guidelines

–

XML Namespace Standards

What Exactly Is EIMS?

A library of additions and extensions to the OAGIS V8 specification:

–

Schema components, elements, attributes and enumerations created for IBM’s reuse

–

Business objects, actions and messaging objects that meet IBM’s business requirements

–

Assembled into EIMS Business Object Documents (EIMS-BOD) that include business

interaction design patterns actions to provide complete interactions (sometimes referred

to as “BODs” in OAGIS)

A set of tools and processes that enable the development, use and

management of EIMS messages

–

Process and tools to create, manage and store and use EIMS-BODs and their

components

EIMS OAGIS V8+ IBM Extensions EIMS-BOD Component Library I-N I-V

What Exactly Does EIMS Provide?

A single set of domain based Business Object Documents and message

transaction design patterns that supports IBM internal business messaging

–Provides business vocabulary stability throughout the enterprise

–Provides a set of standardized message transaction design patterns

–Similar approach used by industry groups (Automotive, Aerospace, Investing, Real estate…)

A single interface development methodology

–Eases specification of requirements (standard data requirements)

–Iterative approach speeds development process (enabling automation of interface creation)

–Reduces specification, design, development & testing – reducing project timelines

A known form that can be transformed in a predictable way

–Enables vocabulary & framework specific transforms

–Empowers applications to control their responsiveness

An architecture that ties together business processes, applications and data

–Service Oriented Architecture (SOA) “services” enabled using reusable message constructs

–Business domain vocabularies (data) become the basis for an enterprise data dictionary

The SOA EIMS “Rapid” Development Process

Current State –> “To Be” State

Test Case Development

SIT & Customer Acceptance Test Service Registration Deployment Plan Develop Qualify Phase Checkpoint Deploy ADD – Arch. Definition Doc. Sequence

Diagrams Deployment Code

& schemas

UDDI, XSR Artifacts Initiative

Definition Concept

Project Scope & Requirements Architecture Topology APD – Arch. Proposal Doc. ECBA Architecture Review Checkpoint Business Transformation Activities EIMS Deliverables (in yellow) Project Nominated Evaluate/ Accept/Reject Business Objectives Sponsorship Leadership Business Process Modeling Business Scenarios

BOD Message Design

Dictionary Design Message Bindings & Definitions Project Definition Scope & Requirements Interface Context Diagrams Interfacing App. Data Structures Detailed Design

CIO Architecture Team – Manage/Develop Corporate Canonical Data Models and Development Process Service Domain Architect – Domain Modeling, UML,

Define Message Scenarios & Data Requirements

Domain Test

Schema Designer

- Designs Transaction Schema (PBI’s) Final Test Cases/Examples Roles and Responsibilities Data Analysis/ Mappings Draft EIMS Schemas Refined EIMS Schemas Message schemas with Fixes EIMS Development

Monitor & Optimize Service Developer Business Operations Enterprise Solution Process Assembler Integration Specialist Flow Composition Messages Entities Business Analysts Information Model Operatio nal Model

EIMS supports the IBM Service-Oriented Development Methodology

EIMS

(Canonical and Transaction Business Objects)

Customer

Supplier

Place Order PurchaseOrder Order

SalesOrder Acknowledge Order Accepted/Rejected OrderStatus SalesOrder Status Receive Order Status PurchaseOrder Status

Change Order Order PurchaseOrder

Confirm Order Change Accepted/Rejected OrderStatus Receive Order SalesOrder Status Receive Order Status PurchaseOrder Status Receive Order SalesOrder

Cancel Order PurchaseOrder Order

Confirm Order Cancel Accepted/Rejected OrderStatus SalesOrder Status Receive Order Status PurchaseOrder Status Receive Order SalesOrder Order Fulfillment SalesOrder/ OrderStatus SalesOrder/ OrderStatus Receive Order Ship Notice Ship Order (unless Cancelled) OrderStatus SalesOrder Ship Notice PurchaseOrder Ship Notice Receive Invoice Send Invoice Invoice SalesOrder Invoice PurchaseOrder Invoice

How the EIMS Architecture supports SOA and Web Services using Message Design Patterns

Initiator: ProcessConfigurationValidation Responder: AcknowledgeConfigurationValidation Used to initiate a “validation” process.

Validate

Initiator: ProcessPurchaseOrder Responder: AcknowledgePurchaseOrder Used to initiate “Process” rules for an existing

object. Process

ProcessPurchaseOrder

Used to synchronize information between application data stores.

Used to cancel a business operation Used to change an object in an application. Used to start a process, resulting in the “creation” of an object in an application/service.

Used to display information. Also used for pub-sub pub-subscriptions (push model)

Used to retrieve information from an

application/service (request-response model).

Description Synchronize Cancel CancelPurchaseOrder Change ChangePurchaseOrder Create CreateServiceRequest Show ShowInvoice Retrieve RetrieveInvoice

Service Operation and Example

Initiator: SyncCustomerParty Responder: ConfirmCustomerParty Initiator: CancelPurchaseOrder Responder: ConfirmPurchaseOrder Initiator: ChangePurchaseOrder Responder: ConfirmPurchaseOrder Initiator: CreateServiceRequest Responder: ConfirmServicerequest Initiator: ShowInvoice

Responder (optional): ConfirmInvoice Initiator: GetInvoice

Responder: ShowInvoice

EIMS Message Interaction Design Pattern

Business Object Document (BOD) message/types

Request/Response Data

input/output

Usage of verb, and the values of its attributes (confirm and acknowledge) operation Service Operation – Request/Response Pair portType/binding Callable Service service

Component (Information System)

EIMS Web Services (optional)

SOA

EIMS Canonical Model – Building Blocks

EIMS Invoice ElectronicCatalog Inspection Bill of Materials Order Quote Invoice Invoice Order Order Bill of Materials Bill of Materials Inspection Inspection ElectronicCatalog ElectronicCatalog Update Inspection Quote Quote Add Quote Show Electronic Catalog List Invoice

Get Bill of Materials Process Order Order Invoice ElectronicCatalog Inspection Bill of Materials Order Quote Invoice Invoice Order Order Bill of Materials Bill of Materials Inspection Inspection ElectronicCatalog ElectronicCatalog Payments Quote Quote Line/Sub-Line Header Tax Document Reference Parties Data Components BOD’s Nouns EIMS Development

EIMS - Layered/Modular Components

EIMS

Invoice

ElectronicCatalog

Inspection

Bill of Materials

Order

Quote

Invoice

Invoice

Order

Order

Bill of Materials

Bill of Materials

Inspection

Inspection

ElectronicCatalog

ElectronicCatalog

Update Inspection

Quote

Quote

Add Quote

Show

Electronic Catalog

List Invoice

Get Bill of Materials

Show Order

IMS Constructs

IMS Invoice ElectronicCatalog Inspection Bill of Materials Order Quote Invoice Invoice Order Order Bill of Materials Bill of Materials Inspection Inspection ElectronicCatalog ElectronicCatalog Inspection Quote Quote Quote Electronic Catalog Invoice Bill of Materials Order IMS Invoice ElectronicCatalog Inspection Bill of Materials Order Quote Invoice Invoice Order Order Bill of Materials Bill of Materials Inspection Inspection ElectronicCatalog ElectronicCatalog Inspection Quote Quote Quote Electronic Catalog Invoice Bill of Materials Order

Reusable Components

Order

Invoice

ElectronicCatalog

Inspection

Bill of Materials

Order

Quote

Invoice

Invoice

Order

Order

Bill of Materials

Bill of Materials

Inspection

Inspection

ElectronicCatalog

ElectronicCatalog

Payments

Quote

Quote

Line/Sub-Line

Header

Tax

Document

Reference

Parties

EIMS Constructs

Order Invoice ElectronicCatalog Inspection Bill of Materials Order Quote Invoice Invoice Order Order Bill of Materials Bill of Materials Inspection Inspection ElectronicCatalog ElectronicCatalog Payments Quote Quote Line/Sub-Line Header Tax Document Reference Parties Order Invoice ElectronicCatalog Inspection Bill of Materials Order Quote Invoice Invoice Order Order Bill of Materials Bill of Materials Inspection Inspection ElectronicCatalog ElectronicCatalog Payments Quote Quote Line/Sub-Line Header Tax Document Reference Parties

Contextual Components/Types

Contextual Components EIMS Constructs

Data Components + Elements

Data Components

Canonical Business Elements EIMS Constructs

Example Use of EIMS – Services in the Same Business Domain

Characteristics

– Requirements are gathered over time.

– Messages are different (e.g., Japan and ASEAN Invoices are 60% the same)

– Services are loosely managed, organization-wise.

Benefits:

– Converge to a common Invoice structure (EIMS Canonical Schema) – simplifies implementation

– Rapid Interface and message design

EIMS Examples TDAccess (Gateway) Japan _IOL (Direct Insite) Asean Invoice Online (IOL) EIMS Invoice EIMS Invoice Business Partners SAP TxHub e2open ESI &GeoHu b FF IGF E2open, TxHub Invoice EIMS Invoice IDOC Invoice EIMS Invoice EIMS Invoice RN Invoice GCPS External Vendors GIL

Global Image Load (GIL) and Global Contract Preparation

System (GCPS)

EIMS Invoice

Electronic or Paper Invoice

EIMS Global Invoice Message

EIMS Global Invoice Message Invoice On Line (IOL) B2B Gateway In te rn e t Customers and Distributors Using a Browser Customers and Distributors Using B2B

5 Internal systems now send out invoices in the

same EIMS format

In the future it will be very easy for any of the internal systems to send to both IOL and B2B

Community Governance Process

- EIMS is a community based development program, (i.e.,

based on open source architecture development principles),

using common architecture principles.

- EIMS business data vocabularies will be

governed/developed by business domain architects based

on the process models required for their domains.

- As EIMS evolves it will contain common, re-usable,

business domain based messaging vocabularies required to

meet the needs of the IBM enterprise.

Projects

Service Domains with proof of concept or deployment projects underway include:

Invoice

Product Availability

Credit Management

Customer Inventory Management

Pricing

Contracts

Finance – Leasing

Opportunity Management

Entitlement

Supply Chain – Product Development/Engineering

Shipments

Sales Order Management

Human Resource Management (Personnel Noun); HR XML – tbd?

Solution Configuration

Major Accomplishments

Rapid XML Schema Development Process, Architecture and Tools are in Place

– Developers and data analysts can quickly create data maps and message solutions.

– The CIO EIMS team trains the Business Unit Architects and programmers when they do their initial SOA/B2B projects

– IBM’s On-line Open Source Library management system is used for Change Control and to manage the EIMS artifacts (using CVS). This system can be used by IBM developers anywhere in the world, 24 x 7, to manage their EIMS

schemas.

EIMS Supports the Roll-out of IBM’s Service Oriented Architecture

– EIMS is the messaging standard for Enterprise Services

– The concept of common message vocabularies has been incorporated into IBM’s SOA methodology.

– Many internal Application to Application projects are also using EIMS.

– The CIO Architecture team is extending EIMS to support UML based development. Important for SOA/ESB deployments.

EIMS Developers Website

– Contains all architecture related development documentation, methodologies, governance processes etc.

– Future: Create an on-line business data dictionary for EIMS messaging artifacts.

EIMS Governance Process is in Place

– The EIMS team has defined a “Service Domain Architect” (SDA) role for Business Areas. The SDA will “own/manage” the EIMS artifacts required to support their Business Units SOA and B2B deployment projects. This will ensure business knowledge and architectural continuity.

Some extensions and deployment lessons.. from the trenches..

Generic BOD - Used where common schema/class required for Web Service

implementations).

Many best practices documents created: CS design, SOA message design patterns, Get Verb

parameters and examples, App Area and Verb usage specifications/rationale

Application Data Exchange metadata - Used in Mediation and gateway environments to

facilitate message hand-offs between environments.

Service Operation Design Patterns - Used to ensure consistency and decrease

implementation costs by re-using similar Service Design Patterns and associated Message patterns.

Use of Confirm[ObjectX] message structure as a generic Service Operation “Response” for

Create, Sync, Change and Cancel Operations.. Required for message orchestration and to facilitate error recovery processing..

Rapid development methodology - UML Activity and Message Sequence Diagrams provide

context for message object and operation definitions.

Governance Model – Domain Architects define their Domain “vocabularies”; Extend OAGIS

artifacts, create new Nouns where there are none etc.

Change Management/Certification – A “hybrid” Cathedral and Open Source “Bazaar”

management model.. Federated (Business Domain) teams follow EIMS architecture principles to design their solutions. Schemas are “certified” prior to implementation to architecture is

Something to “Think” about.. when developing “standards”

Degree of

Acceptance/

Buy-in

Complexity

-Developers, Business Process Owners, Tools Vendors etc.. All have limited time.. Don’t add complexity where it doesn’t provide value!

-- Standards that are difficult to implement will not be accepted.

High Low

Generic Verb

This structure promotes consistency within message schemas by maximizing the reuse of each building block. This shortens the development cycle and learning curve required to create messaging solutions. Example: Consider a PurchaseOrder, which is a commonly used message in eCommerce transactions. The PurchaseOrder message has typical operations (“Actions ) that are performed on it such as “Process”, “Get”, “Show”, and “Acknowledge”, to name a few.

OAGIS currently defines each Verb operation as a separate XML element, which requires a separate message to be created for each Service Operation. That is, although ProcessPurchaseOrder and ShowPurchaseOrder share the common Noun “PurchaseOrder”, two different XML message schemas are required to implement a typical transaction. Implications of this approach:

- For a business interaction that requires two schemas, two separate mapping specifications (used to map the business data for an application to its verb specific XML schema definitions) have to be developed.

- When generating Java code, each schema represents a separate group of code, (e.g., Java package). For the request and response message pattern (typical in Web Services), data has to be moved from one package to the other, which increases development activities, and runtime cost.

- When a new “operation” is required for business needs, a new BOD schema has to be constructed, and a new mapping exercise has to be conducted, which is very time consuming.

Thus, there is a desire to maintain a common Verb-Noun structure in messages, while avoiding the need to develop separate schemas for each business message. To do this requires that the “Operation”

Summary

EIMS:

Standardizes the Message Payload Structures required for consistent Application to

Application Messaging

Provides the Message Objects (Data Vocabulary) used to support ESB Mediation and

SOA Message Interactions

Supports the Deployment of B2B Solutions by standardizing Message Schemas (and

therefore the Services) Required for B2B Interactions.

Promotes, as a Common Design Practice, the use of Reusable Building Blocks for

Constructing Messages.

The EIMS Development Methodology Simplifies Message Construction and Reduces