An Enterprise Service Bus (ESB) is used to provision or mediate services in a corporate network.
It acts as middleware, handling the flow of data between applications. With an ESB, therefore, there is no need to define dedicated communication interfaces between individual applications. Instead, all applications communicate with the bus, which means they have a common, standardized data exchange hub.
An ESB is usually deployed within a modular Service
Oriented Architecture (SOA). In this type of environment, it can handle data communication between a variety of systems such as the customer relationship management, accounting and supply chain management modules. It processes queries from a number of different sources, including business applications, mobile devices, partners and Web browsers. For this, it utilizes a wide array of different services, such as Web portals, Web services or file servers. An ESB makes light work of system integration since there is no need to create custom scripts. So more time is spent making decisions instead of integrating systems. ESBs have been around for a number of
years now. Solutions like IBM WebSphere, Oracle Fusion and webMethods from Software AG all deliver ESB functionality.
These products are highly complex, however, and are best suited to largescale IT
infrastructures. Up to now, ESBs have rarely been deployed in small to mediumsized
networks. This is where Talend’s ESB offering comes in. As mentioned above, the product is based on open source solutions, making it both highly scalable and relatively uncomplicated. As such, it is a viable option for environments that would
previously have been considered too small for an ESB.
We tested: Talend ESB 5.2.1
Enterprise Service Bus
Dr. Götz Güttich Talend Enterprise Service Bus 5.2.1 is an open source, modular solution that allows enterprises to integrate existing or new applications into their IT environment.
In our test lab, we installed, configured and worked with ESB to see how it performed in real time.
Closer look at Talend ESB Talend ESB combines an Eclipsebased tooling
environment with messaging, Web services, intelligent routing and data transformation. Talend enables developers to easily build
Web and data services to quickly integrate heterogeneous IT environments, both on premises, across companies and in the cloud. The services can also be reused across multiple
applications for maximum efficiency. On top of this, Talend ESB comes with failover and load balancing functions.
The Eclipsebased tooling environment allows developers to build, compile, test and deploy Java Web services, REST
applications, data services and messaging routes from a single interface. In addition, developers can design integration solutions based on Enterprise Integration
Patterns (EIPs). EIPs are standard templates used to address
common integration needs.
Talend’s development
environment provides graphical palettes of component and
connector icons. To create new software, all developers have to do is drag the palette icons they require to their workspace and then define the associated parameters. In all, the Talend library has over 450 connectors supporting practically all types of sources and targets for data integration, data migration and data synchronization. These include files, databases, big data solutions and more. It is even possible to insert custom code, if required. We will describe our experience of creating software components later on.
Talend ESB has a centralized deployment console called the
Talend Administration Center. It speeds deployments and
upgrades, allowing
administrators to rapidly respond to integration requests.
Application and data service events are easily monitored with drilldown capabilities into individual service statistics for indepth analysis of interprocess service activity.
Talend ESB is available in three different versions. The Standard Edition is an open source version, the capabilities of which are limited to creating and using services. Talend Open Studio for ESB includes the Eclipsebased development environment.
Talend Enterprise ESB – the solution we used in our test – also has various management functions like a development environment for multiple users. A comparison of the different versions can be found at http://talend.com/products/esb.
Technical background information
Certain Talend ESB components are built on Apache. Web
services, for instance, are based on the Apache CXF framework, which can be used to create SOAP and REST Web services.
For service integration, however, Apache Camel is used. This rule
based routing and conversion engine uses the EIPs mentioned above and is programmed with Java. The runtime container is based on Apache Karaf.
The test
For the test, we installed Talend ESB 5.2.1 on a Windows 8 test PC with 8 GByte RAM, 750 GByte hard disk and an Intel Core Quad processor with 2.4
The Talend Administration Center is a centralized deployment console to manage the ESB solution
GHz clock speed. The Java environment used was Version 7 Update 17 – the most recent one available at the time of the test.
Following installation, we
created a simple “Hello” program with Talend’s Eclipsebased development tool. When users
send their name to this program via the ESB, it returns the string
“Hello “{Name of user}”!”. We then made the program available via the bus.
After that, we submitted a database query via the ESB – again using the development environment. Finally, we
integrated an existing application with the ESB. For this, we used Talend’s free demo car rental application supporting reservation and customer
management functionality. It can be used for all test scenarios once the ESB has been installed.
Installation
Talend ESB 5.2.1 is supplied in the form of an installation file and a depot file, which contains all the necessary components. To install the software, you simply save both files to the same directory, set the JAVA_HOME
variable so that the installer can find the Java environment installed on the system, and start the setup routine. You will then see a wizard displaying the license information. It will first ask for the license file, and then you have to specify the
components you wish to install and their target location. For client configurations, for
example, it often makes sense to install just the development environment. For our test,
however, we decided to install all the components. Administrators also have the option of setting up the server components
(Administration Center,
Command Line, Job Server and Runtime) as system services or as products to be started manually.
To complete the setup wizard, you are asked to configure the database for the Talend
Administration Center and define the SVN server and similar settings.
Getting started
Once installation was complete, we started the Web server manually using Tomcat and the Administration Center plus Karaf as the container. We did this
because we had not set these up as system services in our test environment. Then, with our browser, we went to the page http://localhost:8080/org.talend.a dministrator/. We logged in here using the default login data [email protected]/admin.
For security reasons, the default account only gives you
authorization to manage user accounts. This is how Talend ensures that no standard login data can be used during dayto
day use of the system, and we found this a very sensible precaution. We completed the first step of creating an
administrator account, logged out of the system and then logged in again as administrator.
The administration tool The Talend Administration Center uses a transparent tree structure to give users fast access to the configuration items they are looking for. The
administration tool allows you to define settings for projects and users, view the Dashboard with connections and job analyses, monitor services and manage the servers. As we first needed a server for our test, we created one at this point. To do this, we defined the communication ports, gave the server a name, specified the host and the timeout, etc. As an option, communication with the server can also be secured via SSL. Once we had defined the server, it appeared on screen, immediately displaying its status and the current configuration details. Administrators have the option of switching to the Apache Karaf Web Console by clicking the Admin Server button if they need to. Talend ESB uses Apache Archiva as its repository
management software. When we
Our Hello program and the icons we used
tried it out, we noticed that the installation routine read the path to the Java environment during setup and then entered it in the Archiva file wrapper.conf. If you update your Java installation and the path changes (for example, if you update from Java 7 Update 15 to Java 7 Update 17), it is not enough to reset the
JAVA_HOME variable because the wrapper.conf entry will still be there. If you then try to start Archiva, the system will respond with an error message. The only way to get the system up and running again is to manually change the path in the
wrapper.conf file. A workaround solution for this problem would be to always use the same fixed path to the Java environment.
Talend intends to fix this problem in one of its upcoming software releases.
The Hello application
For the next part of our test, we created the Hello application. We started by opening the Talend development environment and created a new project.
We used the icons mentioned earlier to set up the Hello program. Specific functions are represented by icons in the development studio. They are grouped in a palette on the right
hand side under categories such as Big Data, Business
Intelligence, Cloud, File, Databases, ESB, Custom Code, and so on. When the developer drags an icon from one of these groups and drops it on their workspace, the Talend
Development Studio generates the code automatically in the background, so all the user has to do is define the parameters for the icons.
Let us assume that a user wants to define an Access database as the data source. All they have to do is go to the folder
Databases/Access in the palette and drag the AccessInput component to the workspace.
Then it’s just a matter of entering details like the Access version, the path to the database, the user
name, etc. in the component’s configuration dialog boxes. This completes the configuration process for the data source and the database is now ready to be used. In the next step, the user can for example select or re
arrange particular database entries using a mapping function.
To do this, they just have to drag a mapping icon to the workspace, define the mapping function and create a link between the data source and the mapping icon (by drawing a line between the two icons with the right mouse button).
To finish off, the user just needs to generate a data output (again using an icon) to enable data transformation. Since many of the functions are predefined,
nearly every task can be
accomplished using icons. This reduces manual programming to zero in most cases. As such, even users with little or outdated programming expertise will find the solution easy to use.
For our Hello program, we used a similar approach to that described
for the data transformation demo program. Our objective was not just to transform data, but to communicate with the ESB. The program setup was similar, however. First of all, we created a new service called Hello and imported the WDSL schema.
This ensured that the service metadata was available in our repository so we could reuse it in other components. We then added a new job to the service. This opened the workspace, which already contained two icons. The first was an ESBProviderRequest icon. This receives service requests from the bus and forwards them as appropriate.
The second icon was an ESBProviderResponse component, which sends the response to the bus. Next, we
Once a service has been compiled and made available, the development studio displays the URL of the service in the workspace (here:
http://localhost:8090/services/Hello.World)
dragged an XMLMap icon from the XML folder in the palette and added it to our project. We added a schema for the transferred data (using the WDSL schema data we had previously saved in our repository) and modified the data flow via the mapping function so that every user who sends their name to the Hello service via the ESB receives the response “Hello
“{Name of user}”!”. All we had to do was establish a link
between the request and response components with the mouse and edit the response text so that
“Hello “ is inserted before the input and a “!” string after the input. By now we had completed our program, and so were able to compile it and make it available using the Execute command in the configuration toolbar beneath the workspace.
To use the Hello service, we had to create software that would send the relevant names to the ESB. We did this by creating a new job and adding a
FixedFlowInput icon from the palette. We again added an XMLMap component, plus one ESBConsumer and two LogRow components. The
FixedFlowInput routine generated the input. We could instead have taken a database or similar, but to keep the demo program simple, we decided to define a static table with three names which we created directly during the input icon
configuration process.
The XMLMap component had the job of forwarding the input of the input routine to the service.
We imported the service
information within the mapping function from the repository – it had been saved there when the
service was defined – and linked it with the input from the names table using draganddrop.
The ESBConsumer function then called the service and delivered the response. We had to enter the URL of the service address of our previously created Hello program in the configuration options. The studio provides this as output after compiling the software and making it available in the
workspace. So that we could see something on the screen, we used the two LogRow components mentioned earlier. One of them accepted the response and the
second was used as the destination for any error messages.
Once we had started the job, the three names we had entered in the table and the associated Hello messages appeared in the output space of the development environment. The program therefore worked as expected.
Based on our previous
experiences, we concluded that the software development with Talend ESB is a smooth and easy process for IT professionals in enterprise networks.
Deploying Hello program in a Talend runtime container Once we had verified that our software worked correctly, we set about exporting the program to the Talend runtime container so we could use it independently of the development studio. We rightclicked our Hello service and selected the command Export Service.
A Save as dialog then appeared.
We saved the software directly to the Deploy subdirectory of the Talend runtime container
(directory \esb\container\deploy) to make it available for
immediate deployment. When the export was completed, our Hello service was available in the container.
To test its functionality again, we used the same demo program as before. All we had to do was change the port in the service URL from the test port of the studio (8090) to the default port of the Talend container (8040) in the configuration dialog of the ESBConsumer function. We did this to make sure that the system communicated with the right service. We started the program
Service for checking book titles
again and everything worked just like before, except now we could also view the software output in the output window of the container.
Routing
As mentioned earlier, Talend ESB includes a powerful routing function. This can be used to
distribute inputs to various instances of a service, generate a variety of outputs or filter out duplicate data.
It also supports different software versions used by various partners to send data specifically adapted to the version in question.
However, a detailed description of this functionality would go beyond the scope of this test. We will just mention that routes can be created using the mediation capability of the development studio and they can be
configured using icons like in the software development process.
Routes can also be exported and deployed in exactly the same way as services. Routing is described in great detail in the comprehensive and helpful documentation found on Talend’s website. This can be downloaded for free, as can the two open
source versions of the ESB software.
Data queries
After completing the Hello software test, we turned our attention to the data query job.
Here the user is free to choose any data source – from a database to a small file. For our test program, we used a CSV file containing a number of book titles and the names of the authors.
For communication with the bus, we again used a job with the ESBProviderRequest and ESBProviderResponse icons.
Mapping was again handled by an XMLMap component. For this job, we also added our data source to the program, and connected it to the software using a File/Output FileInputDelimited icon. We then configured the mapping function so that the program compared a book title query received from the bus with the title column in the CSV file and returned the title to the bus if a hit was found. This completed the program so it was ready for network access. Incidentally, the program we used to send the titles to the bus in the test was a specially adapted version of the program we had previously used to send the names to our Hello service.
RentaCar demo
Talend provides a car rental demo application for the ESB.
This software is programmed to manage customers and vehicles and make reservations for specific dates. The code for the application is available in the development studio, but there is also the option of integrating the software as a complete
application in the runtime environment of the ESB. In this test, we chose the latter option since we had already tested service provisioning via the studio.
The data for the demo application is located in the Talend
installation folder under
esb\examples\talend\tesb\renta
car. Apache Maven must be installed on the computer used for the software build process. If this is the case, the program can be installed by executing the mvn clean install command in the RentaCar directory.
When we had completed this step, we set about deploying the demo services in the Talend runtime container. This step is executed via the OSGi command line and is completed after entering two commands. The same applies to deployment of the demo client application. We encountered no hitches
whatsoever when testing the integration of an application in Talend ESB.
Testing the RentaCar demo Once we had installed the Rent
aCar demo, we launched the graphical user interface and used it to simulate a customer
completing a rental transaction.
To do this, we selected a customer account from the customer database when the tool booted and entered the rental date and return date. When we clicked Find, the application called CRMService via the ESB to find the customer data and customer status information. The software then displayed a list of available vehicles to choose from. After that, we clicked Reserve. The application then invoked
The graphical user interface of the RentaCar demo
ReservationService to forward the query. Simultaneously, the
program showed us a
confirmation message with the reservation details.
In addition to a graphical application, the demo also includes a command line
application, allowing the user to search the databases and make reservations. What’s more, the demo can be extended with a Service Locator component and a Service Activity Monitoring (SAM) function.
The Service Locator publicly registers the service endpoints.
This means that service consumers are able to locate endpoints during runtime, without having to know where exactly the endpoint is running.
In addition, the Service Locator component allows service
providers to register or deregister their endpoints automatically and thus make them available to users on demand.
The function of the SAM, however, is to monitor service calls. The data collected can be used to compile usage statistics and pinpoint faults. Users can also run the demo in two
different containers and simulate server failover. In addition, the demo shows how the Web services security feature (WS Security) protects Web services and how repository management software works with the demo.
All of these functions go beyond the scope of this test. They can, however, be tried out with the free versions of the software, which are available to download from the Talend website. They are also comprehensively described in the ESB Getting Started Guide. We would therefore recommend that you install ESB and explore its
performance and functionality for yourself.
Summary
Talend ESB 5.2.1 is relatively easy to get up and running and to manage. We installed the
software in a matter of minutes,
and were instantly impressed with both the clear layout and range of capabilities of the graphical administration console.
The system can therefore be configured and managed with ease. But the main highlight of ESB has to be its development environment and icons, allowing users with little or even no programming expertise to easily create data services and routes.
The Talend ESB development environment does more than just facilitate software development, however. It also provides users with solution integration, testing and deployment support. The other components of the ESB solution, like the Web services stack, load balancing and high availability features, containers and a repository, are equally impressive. All of this makes Talend ESB an excellent choice for enterprises of all sizes thinking about introducing a service bus, but reluctant to go with a product from one of the big players such as IBM or
Oracle. This midsized segment opens up a completely new market for Enterprise Service Buses.
The list of available rental cars
The repository with Archiva
