Enterprise Service Bus

1

Enterprise Service Bus

Daniel Hagimont

IRIT/ENSEEIHT

2 rue Charles Camichel - BP 7122

31071 TOULOUSE CEDEX 7

[email protected]

http://hagimont.perso.enseeiht.fr

2

Intégration - besoins



Briques logicielles (applications)



A gros grain



Distribuées



Technologies différentes (protocoles, systèmes, API …)



Après le développement



Collaboration/Intégration



Communication en réparti



Adaptation des interfaces et des données

3

Problématique



Depuis 20 ans, les DSI se heurtent aux problèmes de



Intégrer des applications hétérogènes



Construire des architectures logicielles complexes



Les maintenir

4

Intégration vs interopérabilité



Définition : L'interopérabilité est la capacité pour un système

d'échanger de l'information et des services dans un

environnement technologique et organisationnel hétérogène

(IEEE, 1990)



L'interopérabilité peut être assurée par



Le développeur (CORBA, RPC, RMI)

5

Intégration point à point



Technologies adhoc (différentes à travers le temps)



The accidental architecture

6

Les ETL (Extract, Transform, Load)



Solution la plus populaire



Exportation des données, adaptation et injection dans

d'autres applications



En mode batch (souvent la nuit)

7

Les EAI (Enterprise Application Integration)



Comme une multiprise



Un connecteur par application

8

ESB (Enterprise Service Bus)



Un EAI décentralisé

9

Ce qui fait un ESB



Un bus (MOM)



Des données (XML)



Des adaptateurs/connecteurs (WS, …)

10

Les technos associées aux ESB



MOM/JMS



Messagerie asynchrone (queues, pub-sub)



Web Services



SOAP, WSDL



XML

11

ESB : les produits



Propriétaires



BEA Aqualogic (acheté par Oracle)



IBM WebSphere Enterprise Service Bus



Sonic ESB de Progress Software



Cape Clear (spinoff de IONA)



OpenSource



Mule



Apache ServiceMix



Jboss ESB

12

Mule



Mule is a Java-based enterprise service bus (ESB) and

integration platform that allows developers to quickly and

easily connect applications to exchange data following the

service-oriented architecture (SOA) methodology. Mule

enables easy integration of existing systems, regardless of

the different technologies that the applications use,

13

What Mule ESB does



Decouples business logic



Location transparency



Transport protocol conversion



Message transformation



Message routing



Message enhancement



Reliability (transactions)



Security



Scalability

14

Mule architecture



Endpoint



Channels for reception



Transformer



Message transformation/enhancement



Router



Message flow control (inbound/outbound)



Service Component

15

Overall view

Generally

MOM + XML

Generally File,

WS, mail,

JDBC ...

16

Mule configuration



Spring based



mule-config.xml



Multiple config files

using import

17

First example

18

Endpoints



Old style



<endpoint address="jms://topic:myTopic"/>



<endpoint address="file://work/incoming?

pollingFrequency=2000"/>



New style (with namespaces)



<jms:outbound-endpoint topic="order.topic"/>



<file:inbound-endpoint path="inbox"

fileAge="1000" pollingFrequency="2000" />

19

Endpoints



Inbound and outbound



Ajax, JDBC, FTP, File, HTTP, JMS, RMI, SSL, TCP, UDP, VM …



Inbound only



IMAP, POP3, Servlet, Twitter …



Outbound only



SMTP

20

Transformers



Default transformers (associated with endpoint)



jmsmessage-to-object-transformer



byte-array-to-string-transformer



Custom transformers

21

Transformers

<jms:jmsmessage-to-object-transformer name="JMSToStringTransformer"/> <xml:xslt-transformer name="XSLT" xsl-file="yourfile.xslt"/>

<custom-transformer name="Custom" class="esb.YourTransformer"/> <jms:inbound-endpoint queue="query.response">

<transformer ref="JMSToStringTransformer"/> <transformer ref="XSLT"/>

</jms:inbound-endpoint>

The default

transformer must be

re-installed

22

Implementing transformers

public class ObjectToXML extends AbstractTransformer { protected Object doTransform(Object payload)

throws TransformerException { try {

StringWriter outWriter = new StringWriter();

IMarshallingContext ctx = BindingDirectory.getFactory( payload.getClass()).createMarshallingContext();

ctx.marshalDocument(payload, "UTF-8", null, outWriter); return outWriter.toString();

} catch (JiBXException e) {

throw new TransformerException(this, e); }

} }

23

Implementing transformers

public class XMLToObject extends AbstractTransformer { private String targetClassName;

protected Object doTransform(Object xmldata) throws TransformerException {

try {

IUnmarshallingContext ctx = BindingDirectory

.getFactory(Class.forName(targetClassName)) .createUnmarshallingContext();

return ctx.unmarshalDocument(new StringReader(xmldata)); } catch (Exception e) {

throw new TransformerException (this, e); }

}

public String getTargetClassName() { return targetClassName;

}

public void setTargetClassName(String targetClassName) { this.targetClassName = targetClassName;

} }

24

Routers



Route message



Inbound : select messages which reach component



Outbound : route messages which leave component



Are applied after (resp. before) an inbound (resp. outbound)

tranformer



Most routers are already there

25

Inbound router example

<inbound> <forwarding-catch-all-strategy> <jms:outbound-endpoint queue="failure.queue" /> </forwarding-catch-all-strategy> <selective-consumer-router> <jxpath-filter pattern="(//resultcode)='success'"/> </selective-consumer-router> <jms:inbound-endpoint queue="list.in" /> </inbound>

26

Common inbound routers

(NB : many routers require customization)

Router name

Description

Idempotent receiver Ensures that only messages are received that contain a unique ID.

Aggregator Combines two or more messages together and passes them on as a single message.

Resequencer Holds back messages and can reorder the

messages before they are sent to the component. Selective consumer Allows to specify whether or not you want to

receive a certain event.

Wiretap router Allows to route certain incoming events to a different endpoint as well as to the component. Forwarding consumer Forwards the message directly to the outbound

27

Outbound router example

<outbound> <list-message-splitter-router> <payload-type-filter expectedType="java.util.List"/> <jms:outbound-endpoint queue="order.queue"> <payload-type-filter expectedType="esb.chapter2.Order"/> </jms:outbound-endpoint> <jms:outbound-endpoint queue="item.queue"> <payload-type-filter expectedType="esb.chapter2.Item"/> </jms:outbound-endpoint> <jms:outbound-endpoint queue="customer.queue"> <payload-type-filter expectedType="esb.chapter2.Customer"/> </jms:outbound-endpoint> </list-message-splitter-router> </outbound>

28

Common outbound routers

(NB : many routers require customization)

Router name Description

Filtering outbound router Routes based on the content of the message Recipient list

Multicast router Sends a message to multiple endpoints

Chaining router Links various endpoints, the result of one endpoint being used as the input of the next endpoint

List message splitter Accepts a list of objects and sends them to different endpoints.

Filtering XML message

29

A component

public class ExampleComponent {

public void processCustomer(Customer customer) { // do something interesting

} }

30

Integration with Spring



Definition of Java object structures in XML

<beans xmlns="http://www.springframework.org/schema/beans"> <bean id="customer" class="org.demo.CustomerServiceImpl">

<property name="anotherProperty" value="someStringValue"/> <property name="customerDAO" ref="customerDAO"/>

</bean>

<bean id="customerDAO" class="org.demo.CustomerDAOImpl"> <property name="name" value="Manning"/>

<property name="address" value="Greenwich"/> <property name="clientnumber" value="12345"/> </bean>

</beans>

31

Integration with Spring

32

33

A full example (1/4)

<property name="targetClassName" value="esb.chapter3.Person" /> </custom-transformer>

<byte-array-to-string-transformer name="BytesToString" /> </mule>

34

A full example (2/4)

35

A full example (3/4)

36

A full example (4/4)

37

Example with message flow (1/8)

JMS to WS wrappers

38

39

Onward flow (3/8)

ISBN data (String)

40

41

Backward flow (5/8)



Return messages



From Amazon

• On queue : amazon.output

• Serialized object (BookQuote)



From Barnes&Noble :

• On queue : barnes.output

• Needs an XML-to-object transformer (with JiBX) to transform into BookQuote object

42

Backward flow (6/8)

43

Custom inbound router (7/8) (aggregator)

public class BookQuoteAggregator extends AbstractEventAggregator { protected EventCorrelatorCallback getCorrelatorCallback() { return new EventCorrelatorCallback() {

public MuleMessage aggregateEvents(EventGroup events) throws AggregationException {

Iterator itEvent = events.iterator();

Collection<BookQuote> quoteList = new ArrayList<BookQuote>(); while(itEvent.hasNext()) {

MuleEvent event = (MuleEvent) itEvent.next(); BookQuote quote = (BookQuote)

event.getMessage().getPayload(); quoteList.add(quote);

}

return new DefaultMuleMessage(quoteList); }

public EventGroup createEventGroup(MuleEvent event,

Object correlationID) { return new EventGroup(correlationID, 2);

}

size of the group create

aggregated message

44

Custom inbound router (8/8) (aggregator)

public boolean shouldAggregateEvents(EventGroup events) { Iterator itEvent = events.iterator();

boolean isAmazonPresent = false; boolean isBarnesPresent = false; while(itEvent.hasNext()) {

MuleEvent event = (MuleEvent) itEvent.next();

BookQuote quote = (BookQuote)event.getMessage().getPayload(); String companyName = quote.getCompanyName();

if("Amazon".equalsIgnoreCase(companyName)) { isAmazonPresent = true; } else if("BarnesAndNoble".equalsIgnoreCase(companyName)) { isBarnesPresent = true; } }

return isAmazonPresent && isBarnesPresent; }};}

public MessageInfoMapping getMessageInfoMapping() { return new MuleMessageInfoMapping() {

public String getCorrelationId(MuleMessage message) {

BookQuote quote = (BookQuote) message.getPayload(); return quote.getIsbn(); } };}} decide if aggregation can be triggered get correlation id (ISBN)

45

Content based routing

<mule>

<xm:xml-to-dom-transformer name="FileToDOM"/> <model name="RoutingExample">

<service name="InsuranceService"> <inbound>

<file:inbound-endpoint path="insuranceInbox" fileAge="500" pollingFrequency="2000" transformer-refs="FileToDOM"/> </inbound> <outbound> <forwarding-catch-all-strategy> <file:outbound-endpoint path="insuranceException"/> </forwarding-catch-all-strategy> <filtering-router> <file:outbound-endpoint path="insuranceCar" outputPattern="car-${DATE}.xml"/> <xm:jxpath-filter pattern="//ins:insurance-type='Car'"> <xm:namespace uri="http://insurance.com" prefix="ins"/> </xm:jxpath-filter> </filtering-router> <filtering-router> <file:outbound-endpoint path="insuranceTravel" outputPattern="travel-${DATE}.xml"/> <xm:jxpath-filter pattern="//ins:insurance-type='Travel'"> <xm:namespace uri="http://insurance.com" prefix="ins"/> </xm:jxpath-filter> </filtering-router> </outbound> </service> </model> </mule> different endpoints

46

Calling a WS and forward

<mule>

<stdio:connector name="stdioConnector" promptMessage="Enter city,country"/> <custom-transformer name="StringToList"

class="org.mule.module.scripting.transformer.ScriptTransformer"> <spring:property name="scriptEngineName" value="groovy"/>

<spring:property name="scriptFile" value="tokenizer.groovy"/> </custom-transformer> <custom-transformer name="ObjectToArray" class="esb.chapter5.transformation.mule.CollectionToArray"/> <model name="TransformationExample"> <service name="WeatherInvokeService"> <inbound> <stdio:inbound-endpoint system="IN"> <transformer ref="StringToList"/> <transformer ref="ObjectToArray"/> </stdio:inbound-endpoint> </inbound> <outbound> <chaining-router> <outbound-endpoint address=" wsdl-cxf:http://www.webservicex.net/globalweather.asmx? WSDL&amp;method=GetWeather"/> <vm:outbound-endpoint path="weather.output"/> </chaining-router> </outbound-router> </service> </model> </mule> WS requires an array Java object a list of cascaded synchronous endpoints

47

Connecting with JDBC

(read)

Table: person

id: long primary key name: varchar[255] processed: boolean <mule> <spring:bean name="datasource" class="org.enhydra.jdbc.standard.StandardDataSource"> <spring:property name="driverName" value="org.hsqldb.jdbcDriver" /> <spring:property name="url" value="jdbc:hsqldb:hsql://localhost/xdb" /> <spring:property name="user" value="sa" />

</spring:bean>

<jdbc:connector name="hsqldb-connector" dataSource-ref="datasource">

<jdbc:query key="get"

value="SELECT * FROM person where processed=false" /> <jdbc:query key="get.ack"

value="UPDATE person SET processed=true WHERE id=${jxpath:id}" /> </jdbc:connector>

</mule>

datasource

48

Connecting with JDBC

(read)



An inbound-endpoint with the "get" query



One message per returned row

49

Connecting with JDBC

(read)

50

Connecting with JDBC

(write)

<mule>

<file:connector name="fileConnector" streaming="false"/> <jdbc:connector name="hsqldb-connector"

dataSource-ref="datasource"> <jdbc:queries>

<entry key="write"

value="INSERT into person (id, name, processed) VALUES(NULL, ${payload}, false" />

</jdbc:queries> </jdbc:connector> </mule>

payload of the message = name

51

Connecting with JDBC

(write)

52

Connecting with SMTP

<file:connector name="fileConnector" streaming="false" /> <model name="mail-model"> <service name="file-to-mail"> <inbound> <file:inbound-endpoint path="chapter6/4a-mail-smtp/in"> <file:file-to-string-transformer /> </file:inbound-endpoint> </inbound> <outbound> <outbound-pass-through-router> <smtp:outbound-endpoint to="mule@localhost" cc="authors@localhost" from="authors@localhost" replyTo="authors@localhost" subject="You've got mail from Mule!"

host="localhost" port="25" user="mule" password="mule" /> </outbound-pass-through-router> </outbound> </service> </model> </mule>

53

Connecting with POP3

<pop3:connector name="pop3Connector" checkFrequency="5000" deleteReadMessages="false"/> <model name="mail-model"> <service name="mail-to-file"> <inbound> <pop3:inbound-endpoint host="localhost" name="mule" password="mule" port="110" /> </inbound> <outbound> <outbound-pass-through-router> <file:outbound-endpoint path="chapter6/4b-mail-pop3/out"/> </outbound-pass-through-router> </outbound> </service> </model> </mule>

54

Connecting with FTP

(write)

55

Connecting with FTP

(read)

<mule>

<ftp:connector name="ftpConnector" validateConnections="true" /> <model name="ftp-model">

<service name="ftp-reader"> <inbound>

<ftp:inbound-endpoint user="bob" password="123password" host="localhost" port="2121"

PollingFrequency="10000" passive="false" path="/" /> </inbound> <outbound> <outbound-pass-through-router> <file:outbound-endpoint path="chapter6/5b-ftp-read/out" /> </outbound-pass-through-router> </outbound> </service> </model> </mule>

56

Business Process Management



Description of an orchestration



In terms of message flow



Statefull



Standard : WS-BPEL (Web Service Business Process

Execution Language)



Jboss



jBPM (Jboss Business Process Management) project



jPDL (jBPM Process Definition Language)

57

Bibliography



General



Enterprise Service Bus:

Theory in Practice (O'Reilly)



Technical



Open-Source ESBs in Action

(Manning)

58

59

60

61