Building an Enterprise Service Bus Using Web Services and Apache Synapse v2

Paul Fremantle

VP of Technology

WSO2

[email protected]

Building an Enterprise Service Bus

Using Web Services and Apache

About Me

ƒ

Co-founder and VP of Technology and Partnerships at

WSO2

¾

The leading Open Source Web services company

ƒ

Co-Chair of the OASIS WSRX Technical Committee

ƒ

Committer on Apache Synapse

ƒ

Member of the Apache Software Foundation

ƒ

Co-author of

Building Web Services in Java 2

nd

Edition

ƒ

http://bloglines.com/blog/paulfremantle

ƒ

Ex IBM Senior Technical Staff Member

¾

developed the IBM Web Services Gateway

Apache Synapse

Synapse is an effort undergoing incubation at the Apache Software

Foundation (ASF), sponsored by the Web Services PMC. Incubation is

required of all newly accepted projects until a further review indicates that the

infrastructure, communications, and decision making process have stabilized

in a manner consistent with other successful ASF projects.

While incubation status is not necessarily a reflection of the completeness or

stability of the code, it does indicate that the project has yet to be fully

endorsed by the ASF.

The Small Print

In other words… an incubator project

Contents

ƒ

What is an ESB?

ƒ

A Web services based ESB

ƒ

Connect, Manage, Transform

ƒ

Apache Synapse

¾

Structure and model

¾

Deployment

¾

Configuration

A Common ESB Definition

“Any to any data connectivity and

transformation (including Web Services)

built on an advanced, proven, reliable

ESB Definition

which means

ƒ

Our existing middleware re-branded as an

SOA platform, with some new web

services adapters at the edges

“Any to any data connectivity and

transformation (including Web Services)

built on an advanced, proven, reliable

So what

is

an

What Is Good about ESBs?

ƒ

SOA based integration

¾

More than just message passing

¾

Metadata about the

contracts

involved

•

e.g.

WSDL, Schema, Policy

ƒ

Bus model is very powerful

¾

Every message goes into the “bus” and ends up

So Why Comments Like This:

ƒ

Anne Thomas Manes:

¾

“I do not believe that an ESB is an essential component

of SOA. When it comes right down to it, you don't need

any new products to do SOA. After all, SOA is about

design, not technology.”

¾

“For the most part, ‘ESB’ is a marketing term.”

¾

“But by 2003, a bunch of other vendors hijacked the

term and applied it to their own integration products.”

Are ESBs a new lock-in point?

Service Oriented Architecture

ESB

ESB

Process

Process

Management

Management

Portal

Portal

Call Center

Call Center

Service

Service

Contract

Contract

Service

Service

Contract

Contract

Service

Service

Contract

Contract

The Only Common Data Model

XML, XML Schema

XML, XML Schema

Process

Process

Management

Management

Portal

Portal

Call Center

Call Center

Service

Service

Contract

Contract

Service

Service

Contract

Contract

Service

Service

Contract

Contract

The Only Common Interaction

Models

HTTP 1.1 200 OK

HTTP 1.1 200 OK

<

<

soap:Envelope

soap:Envelope

>

>

Process

Process

Management

Management

Portal

Portal

Call Center

Call Center

Service

Service

Contract

Contract

Service

Service

Contract

Contract

Service

Service

Contract

Contract

What Do You Need for Enterprise

Integration?

ƒ

Connectivity

ƒ

Data formats

ƒ

Security

ƒ

Reliability

ƒ

Routing

ƒ

Metadata

ƒ

Management

What Do You Need for

Enterprise Integration?

ƒ

Connectivity

¾

Get data and messages from A to B

ƒ

Data formats

¾

Understand what shape the data is in

ƒ

Security

¾

Ensure end-to-end confidentiality, authentication, integrity

ƒ

Reliability

¾

Ensure message delivery

ƒ

Routing

¾

Decouple systems from the physical topology

What Do You Get from Web

Services?

ƒ

Connectivity

¾

Lightweight universal connectivity

¾

HTTP, REST, SOAP, XOP/MTOM (binary data)

ƒ

Data formats

¾

Common data model and data description language

¾

XML, XML Schema

ƒ

Security

¾

Encryption, Authentication, Single Signon

¾

WS-Security, WS-Trust, WS-SecureConversation

ƒ

Reliability

¾

Exactly once in-order delivery

¾

WS-ReliableMessaging

ƒ

Routing

¾

To/From/ReplyTo headers

¾

WS-Addressing

So What Is Synapse?

ƒ

A Web Service intermediary

Ethernet

Ethernet

Synapse.xml

User

JMS

J2EE

Java

The Synapse Pattern of an ESB

Legacy

Gateway

Partner

Axis/C

S

S

.net

CICS

Axis2/Java

_S

JAXWS

Axis2/J

management

routing

transformation

validation

security

policy

DB2

SAP

IMS

PHP Web

A Web Services Mediation Engine

manage

JMX

route

transform

XSLT, E4X

POJO

xp

ath

_re

ge

x

pe

_rf

log

SOAP/HT

TP

auth

java

va

lida

te

SO

AP/

JM

S

MTOM

RES

T/HT

TP

XML/J

MS

SOA

P/W

SS

SOAP/HTTPS

SOAP/WSS

What Does Synapse Do?

ƒ

Connect

ƒ

Manage

Connect

ƒ

Route messages

¾

Based on XPath, Regex,

etc

.

ƒ

Deal with mismatch

¾

Initiate/Terminate RM, WS-Sec

ƒ

Switch

¾

POX or REST to SOAP to JSON

¾

JMS to HTTP to SMTP

ƒ

Virtualisation

Connect Example

<endpoint name=“realService"

address=“http://wso2.com/axis2/services/myRealService”>

</endpoint>

<proxy name=“myVirtualService" description=“virtual

endpoint“ transports="all">

<target endpoint=“realService"/>

</proxy>

Synapse

<proxy/>

<endpt/>

Real

Service

Client

Manage

ƒ

Logging

ƒ

Tracking – adding headers

ƒ

Authentication and Authorisation

ƒ

Schema validation

Manage Examples

<log level=“full”/>

Logs the whole message

<header name=“wso2:logging"

value=“Logged"/>

Adds a SOAP header

Transform

ƒ

XSL Transforms

¾

Apply an XSLT to the SOAP message or Body

ƒ

JavaScript/E4X

¾

E4X is a simple mapping of XML

directly

into

JavaScript

ƒ

POJO

¾

Write Java logic that manipulates the message

Transform Example

<javascript>

<![CDATA[

function mediate(x) {

x..*::price[0] *= 2; // double the price

return x;

}

]]>

Synapse’s Two Approaches

Rules

<endpoint>

<sequence>

Deployment Models

req

prov

HTTP Proxy

Gateway

SOAP

SOAP

SOAP

Deployment

ƒ

Transparent proxy

¾

Configure the client to use Synapse as a proxy

•

Works with or without WS-Addressing headers

¾

Or use a “smart client” and set <wsa:To>

¾

Pure rule based model

ƒ

Gateway

¾

New “Proxy Services”

•

Hosting virtual endpoints in Synapse

•

Either simply direct to an endpoint or

via

a sequence

First Rule of the

Synapse Programming Model

ƒ

“Don’t use the Synapse Programming

Model”

ƒ

The aim of Synapse is to build in the common

functions:

¾

Log, route, modify headers,

etc

.

¾

Engage QoS: WSS, WSRM, WSA,

etc

.

ƒ

And use XML models:

¾

XPath, XSLT,

etc

.

ƒ

Configure the routing and transformation using a

Synapse Config Language

<synapse>

<definitions>

<sequence>…</sequence>

<endpoint>…</endpoint>

<property>…</property>

…

<definitions>

<rules>

mediators

</rules>

</synapse>

Sequences

ƒ

A chain of actions to take on a given

message

¾

The actions may be conditional

Example

<sequence name="stockquote">

<header name="To"

value="http://ws.invesbot.com/stockquotes.asmx"/>

<filter xpath="//*[wsx:symbol='MSFT']"

xmlns:wsx="http://www.webserviceX.NET/">

<makefault>

<code value="tns:Receiver"

xmlns:tns=

"

…

"

/>

<reason

value="Isn't there a Windows API for that?"/>

</makefault>

</filter>

<send/>

</sequence>

Endpoints

A simple local representation of a remote endpoint

<endpoint

name="invesbot"

address=

"http://ws.invesbot.com/stockquotes.asmx"/>

Allows characteristics such as policy, RM, Security to

be configured

Useful Mediators

ƒ

send

¾

Send on

ƒ

drop

¾

Ditch the message

ƒ

log

¾

log message

ƒ

makefault

¾

Send a fault on or back

ƒ

transform

ƒ

header

¾

Set a header in the

message

ƒ

filter

¾

Do this if the XPath or

Regex matches

ƒ

class

¾

Call a user mediator

ƒ

validate

¾

XSD validation

ƒ

switch

Synapse Extensions

ƒ

Packaged as a JAR

ƒ

Uses dynamic discovery to register new XML elements into

the configuration model

ƒ

e.g.

¾

<spr:springmediator…>

•

Allows users to use Spring to wire up a mediator

¾

<js:javascript>…</js:javascript>

•

Allows user to use JavaScript and E4X to implement transforms

ƒ

Extensions use the same programming model as User

mediators

User Mediators

ƒ

Simple Java class that implements the Mediator interface:

boolean mediate(MessageContext mc);

true= continue processing

false= halt processing

Programming Model

(Continued)

ƒ

o.a.synapse.MessageContext

ƒ

A representation of a SOAP infoset

¾

Routing model

•

get/setTo, From, ReplyTo, FaultTo, MessageId,

etc

.

¾

get/setEnvelope()

•

AXIOM representation

¾

isResponse(), isMTOM(), isRest()

¾

get/setProperties()

Asynchronous Internals

ƒ

We are just working on a non-blocking IO

model for Synapse

¾

If full WS-Addressing in place then already

asynchronous

•

Updating Synapse to have same behaviour even the

client or server block on HTTP

¾

Requires a NIO transport for Axis2

•

Since Axis2 does all the IO for Synapse

¾

More important for an intermediary than an

Async IO Diagram

Thread2

Incoming req

Socket open

Thread1

Socket open

Request

processing

Response

processing

Outgoing resp

Outgoing req

Incoming resp

Synapse

Scalable Event Driven

Architecture

ƒ

Simple model of stages and queues for

handling load

“Registry” Support

ƒ

So far everything has been stored in a single

XML file

¾

Synapse.xml

ƒ

Hard to manage

ƒ

Also

¾

Requirement to update the directives without

Registry Model

ƒ

A registry is just a remote (or local) source

of XML files

¾

No special APIs or models from the Synapse

perspective

ƒ

A pluggable model for reading from a

registry

¾

Synapse ships with a URL based implementation

Properties and Dynamic

Properties

ƒ

Properties are just local configuration items

ƒ

Can be used to set up global configuration

ƒ

Dynamic properties are properties backed by

a registry

ƒ

The values are cached and read from the

registry

¾

Allows sequences, endpoints and other config to

Dynamic Property example

<endpoint name=“remote”

address=“http://myserver.com”/>

XML file stored at

http://mystore.wso2.com/endpoint.xml

<!-- load registry values from URLs -->

<registry provider=“URLRegistry”/>

<!-- associate with remote endpoint definition -->

<property name=“myDynamicEP”

key=“http://mystore.wso2.com/endpoint.xml”/>

So What Do You Need?

Anne Thomas Manes again:

ƒ

One or more service platforms (

endpoints

)

ƒ

A SOA management solution (

mediation

)

ƒ

A registry (

metadata

)

ƒ

An XML gateway if services will be exposed outside

the firewall

Synapse can be both the XML Gateway and

the SOA management solution

Publisher

JMS

ActiveMQ

Synapse

Web

Services

Tungsten

Status

ƒ

Incubator in Apache since August 2005

¾

All Apache Contributed code, core committer team

ƒ

M1 release – January 2006

¾

Core function, HTTP Proxy support, XSLT, XPath

ƒ

M2 release – June 2006

¾

Clean up of config language

ƒ

0.90 release –

soon! (we always say that)

¾

Proxy services

¾

Repackage as a MAR

¾

JavaScript/E4X support

Getting involved

HomeWiki / Home site

ƒ

http://wiki.apache.org/ws/Synapse

ƒ

http://incubator.apache.org/synapse/

SVN

ƒ

https://svn.apache.org/repos/asf/incubator/synapse/trunk/java

Try it out

¾ http://incubator.apache.org/synapse/download/M2/download.cgi

ƒ

Submit bug reports

¾ http://issues.apache.org/jira/browse/SYNAPSE