Introduction to
Web Services, SOA, and ESBs
The Enterprise Integration Perspective
Odysseas Pentakalos, Ph.D. Chief Technology Officer SYSNET International, Inc.
Agenda
Enterprise Application Integration Service Oriented Architecture
Characteristics, Advantages, Disadvantages
Web Services
Characteristics, Advantages, Disadvantages Phase I, Phase II for SOA 2.0
Enterprise Service Bus
Characteristics, Advantages, Disadvantages
Business Processes in a SOA
Enterprise Application Integration
Definition from Webopedia:
EAI is the unrestricted sharing of data and business processes throughout the networked applications or data sources in an organization.
Definition from Whatis.com:
EAI is a business computing term for the plans, methods, and tools aimed at modernizing, consolidating, and coordinating the
Demand for Enterprise Integration
A number of business drivers are generating growing demand for Enterprise Integration
Mergers and acquisitions Constant need to cut costs
Constant need to streamline business processes Compliance with government regulations
Gartner just released some statistics around the worldwide application integration market…, finding a seven percent jump in spending from 2004 to 2005. Total revenue totaled $8.5 billion in 2005. Gartner says it looks for a similar growth rate this year in the market.
Requirements
Requirements on an Enterprise Integration solution:
The solution should not cost much; remember the need
for cost-cutting
Should not require extensive changes to existing
applications
Can handle lots of traffic with high reliability
Flexible so that management and reconfiguration is
Enterprise Integration Challenges
Typical environment consists of a collection of
applications
In-house systems: usually developed on the principle of
expediency so extensibility and connectivity options are limited
Packaged-applications: vendors prefer to keep their
Enterprise Integration Challenges
The architect needs to reconcile incompatibilities
and differences at multiple levels:
Business processes are incompatible
Data model variations at the syntactic and semantic
level
Differences in the technologies used for each system Differences in the timing at which systems operate Differences in the interaction patterns: synchronous
Enterprise Integration Approaches
Some of the possible ways of accomplishing integration First taxonomy (from Enterprise Integration Patterns book):
File Transfer
Shared Database
Remote Procedure Invocation
Messaging
Another taxonomy (EIA JavaWorld article by Abraham Kang):
Data-level integration
User interface-level integration Application-level integration
Enterprise Integration Scenario
Loan Broker Scenario; this will be a running
scenario throughout the presentation
Customer Broker Bank Bank 1 3 2 3 3
Service Oriented Architecture (SOA)
The definition of services that encapsulate reusable
business function
The use of explicit implementation independent interfaces
to define services
Loosely coupled; services are independent of other
services
Services are registered in a directory service so that they
can be located during design or runtime
Service definition is not dependent on transport protocol
used to access it
Service can be used in a location/platform/technology
SOA Interaction Pattern
This is the basic interaction pattern in a
SOA-based application:
Consumer Provider Registry 1 : Register 2 : Find 3 : BindService Types
Most people think of services as the request-reply type
Request/Reply – receives information, does something and
returns information
Worker – performs a function or changes state
Monitor – observes a process and generates events
Agent – observes a process and takes action based on
rules
Aggregator – combines results from disparate sources
Process – long running service that implements a business
SOA Vision
Multiple applications utilize the services
Multiple service providers make services available
The lifetimes of the services and applications are fairly
decoupled Composite Application Composite Application Composite Application
Is SOA a new concept?
It is just a new name for a sought-after approach
What about DCE, CORBA, RMI, DCOM?
Lack of industry wide standardization prevented
complete interoperability and platform independence
Transport protocols were proprietary or specific to
distributed object technology
Security concerns and firewalls prevented interaction
across boundaries
Bad timing with respect to availability of computational
Why Web Services for SOA?
Earlier middleware technologies were:
Did not support heterogeneous clients and platforms Could not work over the Internet
Some of the choices were hard to use
Most of the choices were expensive to buy and
maintain
XML: The Data Format
eXtensible Markup Language (XML)
An XML document is based on a hierarchy of
XML Schema
XML language for defining structure and
XSLT: Transformation Language
Overview of SOAP
Definition from SOAP 1.1 Spec
“SOAP provides a simple and lightweight mechanism for
exchanging structured and typed information between peers in a decentralized, distributed environment using XML. ..”
Simplicity is a key requirement
Extensibility is another key requirement Binding into the network is unspecified
Supports various message exchange patterns:
request/respond notification
Structure of a SOAP Message
SOAP envelope SOAP header SOAP body SOAP block SOAP block SOAP block SOAP blockExample SOAP Message
<soap:Envelope xmlns:sn="http://www.sysnetint.com/stockquote/" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" soap:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"> <soap:Header> <h:from xmlns:h=http://www.sysnetint.com/Header> [email protected] </h:from> </soap:Header> <soap:Body> <sn:GetStockQuotePrice> <sn:symbol>MSFT</sn:symbol> </sn:GetStockQuotePrice> </soap:Body> </soap:Envelope>Example Web Service
getQuote(String symbol) <soap:Envelope xmlns: …> <soap:Body> <m:getQuote xmlns="urn:xmethods-delayed-quotes“> <symbol xsi:type="xsd:string">MSFT</symbol> </m:getQuote> </soap:Body> </soap:Envelope> HTTP HTTP <soap:Envelope xmlns: …> <soap:Body> <n:getQuoteResponse xmlns="urn:xmethods-delayed-quotes"> <Result xsi:type="xsd:float">24.67</Result> </n:getQuoteResponse> </soap:Body> </soap:Envelope>WSDL: Introduction
Is an XML document itself with a given schema
Describes the interface to a Web Service
Separates the abstract definition of a service from
the implementation
Describes the what, the how, and the where of
the Web Service
Similar to predecessors such as IDL of CORBA and
WSDL Describes Services
Web Services Description Language (WSDL)
WSDL Platform Tools Client Stubs Server Stubs Client Impl. Server Impl. Requests
WSDL: The description language
Standard for describing services and their
bindings.
<definitions>: is the container and defines name and
namespaces for WSDL document.
<types>: defines any custom data types that are
passed in/out of the service.
<message>: models the data that is exchanged within
method calls
<portType>: operations supported through an endpoint
of a web service
WSDL: Sample document
Demo of sample from W3C
UDDI History
Originally defined for the purpose of providing a
public business and service registry
First few versions came out through UDDI.org
UDDI v3.0 is an OASIS standard
Currently private registries are more popular in
UDDI Basic Concepts
Defines a number of elements as part of the data
model
businessEntity, businessService, bindingTemplate,
tModel
Data Model takes the form of an XML document
Spec defines APIs for inquiry, publication, and
security
The UDDI registry itself is exposed as a set of
UDDI Scenarios
Design time use
Developer searches through the private registry for
existing services that can take part in an orchestrated application
Interest is in the functionality but not the specific instances
Run time use
Application searches through the private registry for the service but focus is now on implementations of it
EAI using Web Services
UDDI Server WSDL WSDL WSDL Query Invoke Query InvokeWeb Services: Phase 1
Services were developed to support a specific purpose Registration of interfaces is not necessary
Discovery and utilization of interfaces is done at design
time
Web Services provide platform independence
Composite Application
Web Services-based Integration
Advantages (Tactical Approach)
Easy to implement Fairly low cost
Does not require enterprise-wide agreements
Disadvantages
Fairly limited in terms of message exchange patterns Lower reusability of the services
Consistency across web services implementations is
SOA Vision
Multiple applications utilize the services
Multiple service providers make services available
The lifetimes of the services and application are fairly
decoupled Composite Application Composite Application Composite Application
WS-* Standards
WS-Addressing – Need a standard approach for
naming endpoints across protocols
WS-ReliableMessaging – Need to support reliable
messaging on top or unreliable protocols
WS-Coordination/WS-Transactions – Need to
support transactions across services
WS-Security – Security is crucial for Web Services
WS-Policy – Need to be able to specify
Why do we need Addressing?
To find a Web Service you need a URL. Simple
right?
Address the web service with the URL, set the
action in the HTTP headers and you are set.
How about intermediaries?
How about SOAP over non-HTTP protocol?
Basic Concepts: Endpoint
References
Endpoint references:
Dynamic generation and customization of service
endpoint descriptions.
Referencing and description of specific service instances
that are created as the result of stateful interactions.
Flexible and dynamic exchange of endpoint information
in tightly coupled environments where communicating parties share a set of common assumptions about
specific policies or protocols that are used during the interaction.
Endpoint Reference Definition
An Endpoint Reference consists of:
Address: is a URI and identifies the address of the
endpoint
ReferenceParameters: XML elements that are necessary
in order to successfully interact with the web service
Metadata: provides an extensible container for
metadata that describes the endpoint
<wsa:EndpointReference
xmlns:wsa="http://www.w3.org/2005/03/addressing">
<wsa:Address>http://example.com/fabrikam/acct</wsa:Address> </wsa:EndpointReference>
Message Information Headers
Message Information Headers: headers to allow
messages to be addressed to an endpoint all
within the SOAP message.
To: The URL of the target service
<wsa:To>http://localhost/MyServiceTest</wsa:To>
From: The ERP of the message’s sender.
Message Information Headers
ReplyTo: The ERP to which the response should be sent
to.
FaultTo: The ERP to which the SOAP fault should be
sent to.
MessageID: Is a URI that uniquely identifies the
message
Action: Takes the place of the SOAPAction
RelatesTo: Element that indicates the MessageID of the
Why do we need Reliable
Messaging?
Network failures may prevent messages from
being delivered
Middleware failures may cause messages from
being delivered
Intermittent failures may cause duplicates
WS-Reliable Messaging Concepts
Sequence: reliable messaging is enforced within
the context of a sequence between two
endpoints
Acknowledgments: an ack from the server
indicates the sequence range of messages
received
Delivery Assurance Policy: endpoints can
Reliable Messaging Model
Application Source Initial Sender Application Source Ultimate Receiver Send Delivery RM Source Transmit RM Destination Receive AcknowledgeReliable Messaging Protocol
Endpoint A Endpoint B Create SequenceCreate Sequence Response (Identifier) Sequence (Identifier, MessageNumber=1) Sequence (Identifier, MessageNumber=2)
Sequence (Identifier, MessageNumber=3, LastMessage) Sequence Ack. (Identifier, Range=1,3)
Sequence (Identifier, MessageNumber=2) Sequence Ack. (Identifier, Range=1..3)
Sequence Message
<wsrm:Sequence ...>
<wsu:Identifier> [URI] </wsu:Identifier>
<wsrm:MessageNumber> [unsignedLong]
</wsrm:MessageNumber>
<wsrm:LastMessage/>?
<wsu:Expires> [dateTime] </wsu:Expires>?
...
Sequence Example
<wsrm:Sequence> <wsu:Identifier> http://fabrikam123.com/abc </wsu:Identifier> <wsrm:MessageNumber>10</wsrm:MessageNumber> <wsrm:LastMessage/> </wsrm:Sequence>Sequence Acknowledgement
<wsrm:SequenceAcknowledgement ...>
<wsu:Identifier> [URI] </wsu:Identifier>
[ <wsrm:AcknowledgementRange ...
Upper="[unsignedLong]"
Lower="[unsignedLong]"/> +
| <wsrm:Nack>[unsignedLong]</wsrm:Nack> + ]
...
<wsrm:SequenceAcknowledgement>
Sequence Ack Example
<wsrm:SequenceAcknowledgement> <wsu:Identifier>
http://fabrikam123.com/abc </wsu:Identifier>
<wsrm:AcknowledgementRange Upper="2" Lower="1"/> <wsrm:AcknowledgementRange Upper="6" Lower="4"/> <wsrm:AcknowledgementRange Upper="10" Lower="8"/> </wsrm:SequenceAcknowledgement>
Create Sequence Message
Request <wsrm:CreateSequence ...> ... </wsrm:CreateSequence> Response <wsrm:CreateSequenceResponse ...><wsu:Identifier> [URI] </wsu:Identifier> ...
Sequence Termination
<wsrm:TerminateSequence ...>
<wsu:Identifier> [URI] </wsu:Identifier>
...
Why do we need Transactions?
In an application based on an orchestration of
Web Services all services involved in an operation
must come to an undisputed resolution.
Typical ACID semantics are useful for some but
long-running transactions need different
transactional support.
Atomic Transactions
Atomicity: on completion either all actions or none
are completed.
Consistency: consistency is the system is
preserved after processing multiple concurrent
transactions
Isolation: intermediate states of the system are
not observable outside the transaction
Durability: if a transaction commits, the changes
Business Transactions
Long-running transactions: transactions may take
hours, days or weeks to complete
Need to be able to select a subset of the original
participants before a commit
Compensating transactions: operations that are
Standards and scope
WS-Coordination:
allows a distributed system to establish a group of participants
around an activity
participants can register interest in participating in the outcome Coordination protocol can vary but operates upon completion of
the activity
WS-Atomic:
Handles ACID transactions and defines variations of 2PC
WS-Business Activity:
Allows for the definition of nested scopes of operations Allows for the definition of compensating transactions
Why do we need Security?
Protect messages from someone eavesdropping
on the network
Prevent tampering with the messages
Currently security for web services is done at the
transport layer (SSL).
This won’t work when the same message has
multiple links to traverse
WS-Security Specifications
Authentication Profiles
Username/password Kerberos X.509 CertificatesMessage Integrity
XML Signature XML Key ManagementMessage Confidentiality
XML EncryptionSOA for Enterprise Integration
They all need to talk to one another despite running on top ofdifferent protocols
They need support for all the WS-* services we talked about The data structures between the elements may not be identical Need to be able to support various message exchange patterns
Service Service
Service
Service Service
Service
Enterprise Service Bus (ESB)
Abstract the development of service components from wire
protocol and deployment platform from below and the business processes above
Facilitates the requirements of a highly-scalable,
fault-tolerant, message-driven, service-oriented enterprise.
Utilizes XML and related-technologies for unifying the
message format
Provides transformation and routing capabilities to the bus
as a service
Provides centralized management capabilities of the
ESB Architecture
Provides a standards-based bus and adapters for multiple transports and protocols along with common services
Java Business Integration
Business Processes
Composition of services into a business process
Client Travel Reservation United Airline Reservations Delta Airline Reservations Hertz Car Reservations Hotel
BPEL
Business Process Execution Language:
Language for describing business processes
Convergence of a number of earlier efforts (WSFL,
XLANG)
Invoke operations in synchronous/asynchronous mode Invoke operations sequentially or in parallel
Define fault and transaction handling
Loan Broker with BPEL
Broker is defined as a BPEL document
Customer BPEL Process Bank Bank Bank 1 3 2 3 3
Enterprise Integration Design Patterns
Design patterns for Enterprise Integration
Fill the gap between high-level architecture and
implementation Pattern Classification
Messaging Systems: basic concepts in integration with messaging Messaging Channels: describe channels used for integration
Message Construction: describe message types and intent
Message Routing: describes provision of routing and brokering Message Transformation: variations of message translator
Messaging Endpoints: connecting to and from the messaging system System Management: management of the integration infrastructure
Sample EI Patterns
Messaging Channels
Datatype Channel: all
messages on a given channel are of the same type
Invalid Message Channel:
what to do about invalid messages.
Dead Letter Channel: what
to do about undeliverable messages.
Sample EI Patterns
Messaging Routing
Content-based Router:
routes message based on the message’s content
Recipient List: routes
message to dynamic list of recipients
Splitter/Aggregator: one
splits message into sub-messages; other one
Application of EIP
Loan Broker using Enterprise Integration Patterns
Customer
Credit Agency
Endpoint
Data Enricher
Mule EIP Server
Mule is an open source framework based on the concept
of an ESB (mule.codehaus.org)
What’s with the name?
Follows an event-driven architecture
System consists of producers, consumers and a message server Configuration of the routing of events between components takes
place outside the scope of consumers/producers
Container follows the SEDA architecture model
Master’s thesis work of Matt Welsh at UC Berkeley
Conclusion
SOA has great potential for finally achieving reusability Web Services is the platform that will make SOA a reality Existing holes to achieve the full vision are currently in
development or production
An Enterprise Service Bus provides the glue to make SOA a
Mule Configuration
An XML file defines the integration model:
connector: Configuration of the components that attaches
to the various providers
container-context: Configuration of the container that will
provide the components
endpoint-identifiers: Configuration of endpoints that
provide services
interceptors: code to execute before and/or after a
component
model: Configuration of the components (UMO) that
Mule-based Loan Broker
Loan Broker Credit Agency Gateway Credit Agency Lender Service Banking Gateway Bank 1 Bank 2 Bank nService Definition
Loan Broker: (REST Web Service) receives requests for
loan quotes
Credit Agency: (EJB Service) provides credit information
about the customer
Lender Service: (Web Service) receives customer loan
requests with credit information and forwards requests to appropriate banks
Banks: (Web Service) provide quotes on customer loan
Endpoint Definition
<endpoint-identifiers> <endpoint-identifier name="LoanBrokerRequestsREST" value="jetty:rest://localhost:8080/loanbroker"/> <endpoint-identifier name="LoanBrokerRequests" value="vm://loan.broker.requests"/><endpoint-identifier name="LoanQuotes" value="jms://esb.loan.quotes"/>
<endpoint-identifier name="CreditAgencyGateway" value="jms://esb.credit.agency"/> <endpoint-identifier name="CreditAgency"
value="ejb://localhost:1099/local/CreditAgency?method=getCreditProfile"/>
<endpoint-identifier name="LenderGateway" value="jms://esb.lender.service"/> <endpoint-identifier name="LenderService" value="vm://lender.service"/>
<endpoint-identifier name="BankingGateway" value="jms://esb.banks"/>
<endpoint-identifier name="Bank1" value="axis:http://localhost:10080/mule"/> <endpoint-identifier name="Bank2" value="axis:http://localhost:20080/mule"/> <endpoint-identifier name="Bank3" value="axis:http://localhost:30080/mule"/> <endpoint-identifier name="Bank4" value="axis:http://localhost:40080/mule"/> <endpoint-identifier name="Bank5" value="axis:http://localhost:50080/mule"/> </endpoint-identifiers>
Loan Broker Definition
<model name="loan-broker">
<!-- The loan broker is used to receive loan requests --> <mule-descriptor name="LoanBroker"
implementation="org.mule.samples.loanbroker.esb.LoanBroker"> <inbound-router>
<endpoint address="LoanBrokerReqREST“ transformers=“RestReqToCustReq”/> <endpoint address="LoanBrokerRequests"/> </inbound-router> <outbound-router> <router className="org….outbound.OutboundPassThroughRouter"> <endpoint address="CreditAgencyGateway"/> </router> </outbound-router> <response-router timeout="1000000"> <endpoint address="LoanQuotes"/> <router className="org…..routers.BankQuotesResponseAggregator"/>
Credit Agency Definition
<!-- The credit agency service will get the credit profile for a customer --> <mule-descriptor name="CreditAgencyGateway" implementation="org.mule.components.builder.ReflectionMessageBuilder"> <inbound-router> <endpoint address="CreditAgencyGateway"/> </inbound-router> <outbound-router> <router className="org.mule….FilteringOutboundRouter"> <endpoint remoteSync="true“ transformers="LoanQuoteRequestToCreditProfileArgs" responseTransformers="CreditProfileXmlToCreditProfile“ address="CreditAgency"/> <endpoint address="LenderGateway"/> </router> </outbound-router>
