openadaptor 3.2
Whitepaper
What is openadaptor?
Openadaptor is a Java software toolkit for Enterprise Application Integration. It was written to help a large financial organisation integrate it’s applications with Message Oriented Middleware. The toolkit provides a framework and components for building links between systems and various middleware solutions, what we call an adaptor in this case.
The toolkit provides software components for connecting to a variety of transports (e.g. JMS, Tibco, RDBMS, SOAP, HTTP, SMTP, Files). It provides components for “working with” different data representations (e.g. XML, JDBC result sets, fixed width and delimited string records); however it is not tied to any specific data
representation.
The framework includes support for transactions (local or JTA), the ability to capture and manage processing exceptions and remote administration (JMX or HTPP). The components can be quickly “wired together” and allow a developer to deliver a production quality adaptor extremely quickly.
The framework is designed to be as lightweight as possible it does not require any infrastructure other than the Java Runtime Environment. The framework can be used to run adaptors as standalone process, embedded in other software or deployed within application containers. The components can be used with or without the framework and writing custom components does not require extensive knowledge of the framework.
All openadaptor components are Java Beans and can be used with the spring framework (http://www.springframework.org). This means that adaptor processes can be developed by writing an XML configuration file rather than having to write and compile Java code.
Give me an example?
The possibilities for using openadaptor are broad. It can be used to link enterprise systems to a core middleware solution or it can be used to link systems together directly. It can even be used to do simple file transformation. Its flexibility reflects the technological diversity of the large organisations in which it is used.
For example you may want to poll an application database for events, extract the data that relates to those events, transform the data to an XML document and publish to a JMS topic. openadaptor has components to do all of this and this simple adaptor can be assembled by configuring the components and wiring them together in a simple “Pipeline”. JDBC Read Connector XML Converter JMS Write Connector Application Database JM S
All of this can be achieved by writing a short XML configuration file. We provide a standard program to run the configuration as a stand alone process. However you are free to “wire the components together” using a few lines of Java code or to embed the XML configuration in any other process that uses the spring framework. Functionality can be added to the adaptor by inserting additional components in the “pipeline”. For instance you may want to filter out certain data, transform it into your company’s enterprise business model and perhaps validate it before publishing it.
JDBC Read Connector XML Converter JMS Write Connector Application Database JM S
Filter ProcessorXSLT ValidatorXML
The adaptor can easily be system tested, by swapping out the transport specific connectors with file based connectors. When the components are used with the framework you automatically receive the benefits of the framework, for example standardized process management and remote administration.
Design
The three fundamental components in openadaptor are as follows…
Read Connector These connect to an external resource and receive / poll data
Processor These operate on data, either by transforming it in someway or discarding it
Write Connector These connect to an external resource and send / publish data.
Logically an adaptor looks like this… Read
Connector Object[] Processor Object[] ConnectorWrite
External Resource A External Resource B
In that every adaptor will have one or more read connectors, zero or more
processors and one or more write connectors. These are typically chained together in a “pipeline”. The data that “flows through the pipeline” is an array of “Plain Old Java Objects”.
The simplest adaptor it is possible to configure looks like this… Read
Connector ConnectorWrite
External Resource A External Resource B
Processors components can be chained together like this…
Read Connector Processor 1 Write Connector External Resource A External Resource B Processor 2 Processor 3
As stated above, an adaptor can have multiple write connectors, like this… Read Connector Processor1 Write Connector 1 External Resource A External Resource B Processor 2 Write
Adaptors can also have multiple read connectors, like this… Read Connector 1 Processor Write Connector External Resource A External Resource B Read Connector 2 External Resource C
Adaptors can also have multiple distinct “pipelines”, like this… Read Connector 1 Processor 1 Write Connector 1 External Resource A External Resource B Processor 2 Write Connector 2 External Resource D Read Connector 2 External Resource C
For simple adaptors (see example spring configurations and code) a developer only needs to deal with Connectors and Processors and specify how they are “chained together”. The standard framework Adaptor and Pipeline components will “auto box” the Connectors and Processors in the appropriate adaptor framework components. The rest of the section provides more details about the adaptor framework.
Within the adaptor framework the Connectors and Processors are wrapped inside Node components. These provide the lifecycle management, exception capture, transaction management and data batching capabilities of openadaptor. There are specific subclasses of Node for the different fundamental components.
…wraps…
Read Node Read Connector (and optionally a Processor)
Processor Node Processor
Write Node Write Connector (and optionally a Processor) Where as the fundamental components (Connectors and processors) are data agnostic, the adaptor components expect to receive data wrapped in a Message object and respond with a Response object. These are used to propagate context information such as the transaction and the actual data, discarded data and processing exceptions.
Nodes are typically managed by an Adaptor component. This co-ordinates the lifecycle and allows Adaptor Read Nodes to initiate the processing of messages. The following diagram illustrates a typical adaptor configuration.
Processor Node Write Node
Read Node
Read
Connector Processor ConnectorWrite
Message / Response Message / Response External Resource A External Resource B Processor
(optional) Processor(optional)
Adaptor
Pipeline / Router
Message / Response Message / Response
A Pipeline / Router is a component that “knows” how the Nodes are chained together. A Pipeline is actually a simple case of a Router. It can be used when the linkage between the adaptor components is a non-branching chain and there are no complex exception capture and routing requirements. The advantage of using a Pipeline is that it requires less code (or configuration if you are using the spring framework) to assemble. The Router allows more sophisticated routing and exception
management.
The Adaptor delegates the message routing to a Pipeline or Router. But it is worth noting that all the adaptor components (Node subclasses, Adaptor, Router and Pipeline) actually implement the same interface (IMessageProcessor). So it is perfectly legitimate to couple Nodes together directly or for an Adaptor to delegate directly to a Node. You are also free to develop complimentary frameworks which use Nodes, Connectors, Processors, Routers and Pipelines.
Key interfaces and classes
The components described in the previous section are illustrated in the following class diagram. org.oa3.core.node.Node org.oa3.core.adaptor.Adaptor org.oa3.core.router.Router org.oa3.core.IDataProcessor Exception[] validate() reset() Object[] process(Object) org.oa3.core.lifecycle.IRunnable int getExitCode() org.oa3.core.node.WriteNode org.oa3.core.node.ReadNode org.oa3.core.IMessageProcessor Response process(Message) java.lang.Runnable extends org.oa3.core.router.Pipeline org.oa3.core.IWriteConnector connect() disconnect() Object deliver(Object[]) org.oa3.core.IReadConnector connect() disconnect() isDry() Object[] next(long) Object getReaderContext() implements implements may have a has a has a extends has a has a implements implements extends implements
has one or more
org.oa3.core.transaction.ITransactionInitiator
extends
The following interfaces and classes are core to lifecycle management.
ILifecycleComponent validate(List) start() stop() boolean isState(IState) waitForState(IState) addListener(ILifecycleListener) removeListener(ILifecycleListener) org.oa3.core.lifecycle.Component org.oa3.core.IComponent id extends implements implements
Distribution and Code Organisation
The distribution of openadaptor contains jar files, example code and documentation. There are a number of jars
openadaptor.jar The openadaptor code
openadaptor-depends.jar The single jar of the core third-party code upon
which the openadaptor code depends
openadaptor-spring.jar The openadaptor code that is complimentary to
the spring framework
The example code area contains readme files that describe how to correctly setup the java classpath.
The package structure of openadaptor is split as follows
org.openadaptor.core Contains core interfaces and implementations
org.openadaptor.auxil Contains connectors and processors that rely
on the core java and javax packages. Processors that convert between data representations are referred to as convertors.
org.openadaptor.thirdparty contains connectors and processors that rely
on third-party software
org.openadaptor.spring Contains components that directly use the
spring framework
Why the re-write?
Openadaptor was originally developed in 1997, this predated the popularity of XML and integral to openadaptor was a canonical data representation called DataObjects. DataObjects have long been regarded as non-strategic.
Certain standards and technologies have evolved that we wish to accommodate. Using components of openadaptor outside the adaptor paradigm is difficult as is extending it to provide more sophisticated functionality such as workflow.
