Attunity Integration Suite

(1)

Attunity Integration Suite

A White Paper

February 2009

(2)

Attunity Integration Suite

Attunity Ltd. follows a policy of continuous development and reserves the right to alter, without prior notice, the specifications and descriptions outlined in this document. No part of this document shall be deemed to be part of any contract or warranty. Attunity Ltd. retains the sole proprietary rights to all information contained in this document. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopy, recording, or otherwise, without prior written permission of Attunity Ltd. or its duly

appointed authorized representatives.

Attunity and Attunity Connect are registered trademarks, and Attunity Federate, Attunity Stream, Attunity InFocus, the Attunity logo, Application Adapter Framework, Attunity AAF, are trademarks of Attunity Ltd. All other marks are the property of their respective owners.

(3)

1. INTRODUCTION

Over the last 30 years companies have invested heavily in using information technology to optimize business operations, building home-grown systems or deploying specialized software packages such as MRP, ERP and CRM. These systems maintain core assets for the business:

business data and business logic. Increasing competitive and regulatory pressures strengthen the need to reuse and repurpose these assets and integration software has evolved to answer this need.

Leading organizations continue to standardize their integration practices and software portfolio to make integration easier and reduce its costs. This is evident in the creation of Integration Competency Centers and in the adoption of Service Oriented Architectures (SOA) and choice of integration platforms including EAI (Enterprise Application Integration), DI and ETL (Data Integration and Extract, Transform, and Load).

As companies design solutions to address these needs, they run into the barriers of integration, especially when dealing with legacy systems, including non-standard data sources and

applications that have not been designed for interoperability. These barriers, sometimes referred to as ‘the last mile of integration’, have a significant impact on the ability to deliver solutions on time and on budget since they require domain expertise and skill sets that are not common, require turning non-standard data into standard and modern formats, and are commonly underestimated and under-budgeted. These barriers often come up in initiatives such as:

 Business Intelligence (BI) and Data Warehousing (DW)

 Data Integration and Synchronization

 Application Integration (EAI) and Service Oriented Architecture (SOA)

 Enterprise Portals

 Legacy Modernization

Attunity is a provider of integration software that delivers real-time data integration and event capture across many data sources data sources and applications on platform ranging from Windows to the Mainframe. The Attunity Integration Suite enables organizations to establish a foundation for rapid and standard integration, reduce the risks of legacy integration, and reduce total cost of ownership (TCO) by relying on a single cross-platform architecture that allows existing systems to be reused in many different initiatives.

This white paper discusses how enterprises can take a strategic approach while dealing with tactical data integration needs. It then provides an overview of the Attunity Integration Suite (AIS) and how companies can use it to remove the barriers between business users and enterprise information.

(5)

2. DATA INTEGRATION – STRATEGIC AND TACTICAL NEEDS

Many organizations take a tactical approach when dealing with data integration, that is – address a specific challenge at the time when it is encountered. This is now changing as companies are taking more of a strategic view with the purpose of putting in place the right infrastructure that can address current and future initiatives as well as business and IT trends. A favorable approach is to address tactical challenges strategically, that is – solve the most

immediate pain while using a foundation that can be applied and extended to other initiatives over time.

The reason that data and application integration needs to be viewed strategically is evident when one looks at the number of initiatives and projects that require integration including initiatives that:

1. Provide business users with greater visibility to improve operations, productivity and control 2. Automate and streamline business processes

3. Integrate enterprise data for integrity and efficiency

The first type of initiative includes projects such as Enterprise Portals, Business

Intelligence, and Corporate Performance Management, which deliver information to business users. These projects require that data from enterprise sources, including operational systems, be readily available, with lower and lower latency requirements.

The second type of initiative includes projects such as Enterprise Application Integration (EAI) and Business Process Management (BPM) that require integration with existing applications and their embedded business logic, as well as with enterprise data and events.

The third type of initiative deals with propagating, synchronizing, and moving data between enterprise systems to create a single version of truth from various sources, to maintain integrity, and to make low-latency information available for decision support.

Finally, organizations are also looking at modernizing their legacy systems, to increase their value and extend their use. This includes service oriented architectures and web integration to existing mainframe and other applications.

All of these initiatives commonly deal with the following barriers:

 How to access information in non-relational, legacy data sources? (e.g. VSAM, IMS, RMS)

 How to interoperate with legacy applications not designed for integration? (e.g. CICS, RPG)

 How to access a single view of information from multiple data sources?

 How to move data efficiently and in real-time, given the growth in data volumes?

(6)

These many different initiatives require data integration and an IT strategy that can be used to ensure a common infrastructure be used to reduce costs, risks, and increase reusability and ROI.

All of these initiatives require three building blocks of data integration:

1. On-Demand Data Access – seamless and standard access to enterprise data 2. Virtual Data Federation – on the fly Enterprise Information Integration (EII) 3. Change Data Capture – enabling real-time and efficient processing of data

The Attunity Integration Suite (AIS) provides a modular solution that allows organizations to address various tactical needs quickly, while relying on a comprehensive platform that supports common strategic goals including reusability and extensibility. The following section provides an overview of AIS including the three products in the suite – Attunity Connect, Attunity Federate, and Attunity Stream.

(7)

3. ATTUNITY INTEGRATION SUITE

The Attunity Integration Suite (AIS) is a comprehensive integration platform for on-demand access and integration of enterprise data sources and legacy applications. The suite is based on proven server architecture, running on many platforms ranging from Windows and Unix, to the Mainframe, and provides enterprises with a foundation for removing integration barriers. The suite includes many integration services under the banner of three products:

The following sections provide an overview of each product in the suite.

(8)

3.1. Attunity Connect

Attunity Connect provides standard access to enterprise data sources and legacy applications It includes the following key capabilities:

Data Connectivity (SQL and XML based)

Key capabilities:

Provides many SQL clients (32 and 64bit) with a solution that fits any enterprise environment including ODBC, JDBC, OLEDB and ADO.NET.

Universal Data Access

Supports ANSI SQL-92 to allow seamless integration and enables the reuse of a common skill set.

Standard SQL support

Provides metadata import facilities and catalog definitions enabling clients to use SQL to access non-relational sources such as VSAM and Adabas.

SQL Access to Non-Relational Sources

Supports reading as well as update capabilities to all supported data sources.

Read and Write

Supporting 2PC interfaces allowing clients to include data sources in distributed and global transaction.

Transaction Management

Provides a mechanism to protect against ‘bad queries’ and its impact on the underlying data source.

Query Governing

(9)

Attunity Connect provides SQL-based data access to data sources on many platforms including:

 Mainframe: VSAM, IMS, Adabas, DB2

 HP NonStop (Tandem): Enscribe, SQL/MP

 AS/400: DB2/400

 OpenVMS: RMS, Oracle Rdb, Oracle ,DBMS, Adabas

 UNIX: Adabas, C/D-ISAM, Oracle, DB2, Sybase, Informix, generic ODBC.

 Windows: Adabas, D-ISAM, SQL Server, Oracle, DB2, Sybase, Informix, generic

Application Adapters and Service Oriented Integration

Key capabilities:

Provide virtual, service-oriented integration components (adapters) that allow connectivity to seamlessly interoperate with legacy programs using XML, standard APIs and Web Services.

Service-enabling legacy programs

Provide virtual, service-oriented integration components (adapters) that allow connectivity to package data operations as reusable services, accessible using XML, standard APIs and Web Services.

Service-enabling data sources

(10)

Every adapter defined in Attunity Connect is automatically accessible using XML, standard APIs including JSA and .NET, as well as Web Services (i.e.

SOAP, WSDL).

Universal Standard Interoperability wizards

Generate events from within legacy applications, automatically transforming them into XML messages that can be delivered to enterprise applications.

Event generation

Supports XA interfaces enabling adapters to participate in a distributed or global transaction.

Transaction management

Attunity Connect provides XML/Service-based integration to legacy programs on many platforms including:

 Mainframe: CICS, IMS/TM, COBOL

 HP NonStop (Tandem): Pathway, COBOL

 AS/400: RPG, COBOL

 OpenVMS: COBOL, Basic

 UNIX: Tuxedo, C and any other 3GL

 Windows: Tuxedo, C, COM Applications and any other 3GL

Ease of Use

Attunity Connect does not require any coding and users leverage the Attunity Studio graphical user interface to quickly configure data sources access and adapters. The Attunity Studio facilitates the set up, configuration and testing by providing:

 Metadata import wizards

 Metadata configuration and management tools

 Query Builder tool for testing queries

 Integrates Test facility for adapters

 Server configuration wizards

(11)

3.2. Attunity Federate

Attunity Federate provides on-demand virtual data federation for enterprise information integration (EII). The purpose behind EII is to allow the definition of virtual views across enterprise data sources, which enables the simplification of data models used by enterprise applications, developers, and business analysts. By using these models, information is extracted on the fly from the data sources without the need to move the data and integrate it in a

secondary data store.

Attunity Federate can build virtual views across all the data sources supported by Attunity Connect and includes the following key capabilities:

Create new, virtual data sources, whose catalog includes tables from different data sources. This enables applications to access a single data source and retrieve relevant tables, without knowing their underlying data store, location, or structure.

Single Virtual Catalog across Sources

Create virtual views that join together data from heterogeneous sources, presenting the user with a simplified and integrated data model.

Creating Single Virtual Views

Attunity Federate allows applications to issue queries that access information from

heterogeneous sources. It processes and optimizes the execution of such queries.

Dynamic Federated-Query Processing

(12)

Attunity Federate includes a powerful and Distributed Query Processor (DQP) and Optimizer. These work together to execute federated queries in the most efficient manner, while delegating processing to agents that reside close to the data source.

Distributed Query Processing and Optimization

All the virtual views/databases defined in Attunity Federate are accessible using all the supported SQL and XML interfaces in the platform.

Standard Access via SQL and XML

This utility allows users to analyze the execution path of a federated query for optimization purposes.

Query Analyzer

The following diagram illustrates the concept of virtual data federation and query processing:

(13)

Ease of Use

Attunity Federate does not require any coding, and users leverage the Attunity Studio graphical user interface to quickly define federated databases and single views. The Attunity Studio facilitates the set up, configuration and testing by providing:

 Visual and query-based editors for defining virtual views

 Reusability of all metadata defined by Attunity Connect

 Metadata configuration and management tools

 Query Builder tool for testing queries

3.3. Attunity Stream

Attunity Stream captures changes to enterprise data sources and delivers them to enterprise applications in batch or real-time, allowing low-latency and efficient data movement. It provides a way to deal with the need for low latency, growing data volumes, and shrinking batch windows.

Attunity Stream includes the following key capabilities:

Attunity Stream’s CDC Agents are non-intrusive and interface with database logs and journals to reduce the impact on operational data sources.

Non intrusive change capture

Attunity Stream automatically adds to every captured record relevant metadata information that helps processing downstream, including the timestamp and the type of the change (i.e. insert, update delete)

Change Metadata Information

(14)

Attunity Stream creates a staging area into which changes are offloaded from the data server, reducing impact to a minimum.

The staging area creates a new ‘change source’ that applications can use to retrieve only changes to the data.

Change Source (staging area)

Changes can be retrieved periodically or continuously by many applications, where each maintains a ‘position’ within a stream of changed records.

Change Streaming

Stream positions are persisted, such that Attunity Stream can restart/recover from any failure and continue to process changes while guaranteeing their delivery.

Reliable Delivery and Recovery

ETL tools interface with using standard SQL interfaces that

leverage common skill sets and practices in the ETL space. Attunity Stream was tested and proven with all the leading ETL vendors.

ETL integration

EAI tools can retrieve changes from Attunity Stream to process them in a more message/event oriented manner.

EAI integration

Attunity Stream provides CDC for the following data sources:

 Mainframe: CICS, IMS/ TM, Adabas, DB2

 HP NonStop (Tandem): Enscribe, SQL/MP

 AS/400: DB2/400

 UNIX: Oracle (9i, 10g, 11g), Adabas

 Windows: SQL Server (2000, 2005), Oracle (9i, 10g, 11g), Adabas

Ease of Use

Attunity Stream does not require any coding, and users leverage the Attunity Studio graphical user interface to set up CDC solutions. The Attunity Studio facilitates the set up, configuration and testing by providing:

 CDC solution guides

 Metadata import and configuration wizards

 CDC configuration wizards

(15)

4. TYPICAL USAGE SCENARIOS

The Attunity Integration Suite can fit many integration scenarios. This section highlights common and typical use cases, related applications or projects, and how AIS simplified the access and integration.

4.1. SQL Connectivity

SQL is a known skill set for application developers and a common interface that applications know and use in order to retrieve data. While modern databases provide SQL connectivity out of the box, legacy data sources and file systems do not. This is where Attunity Connect removes the barrier, by making such data sources appear relational and allowing the data to be accessed using standard SQL.

Typical applications that require SQL connectivity include:

 Reporting tools – for designing and providing reports to business users

 ETL tools – for bulk loading of source data

 J2EE or .NET applications

Typical usage scenarios that use Attunity Connect include SQL Connectivity to Adabas, VSAM, IMS/DB, RMS, Enscribe, and ISAM.

4.2. Application Connectivity

XML has become the expected way to interface with applications, preferably by using standard APIs or Web Services. While modern packaged applications have started to offer open interfaces, legacy applications were not designed to do so and interfacing with their embedded business logic is complex and challenging. Attunity Connect removes this barrier by defining virtual services on top of these legacy applications that provide seamless

interoperability.

Typical applications that require Service/XML-based application connectivity include:

 EAI tools -for invoking business logic as part of an automated process

 J2EE or .NET applications – that need to reuse existing business logic

 Legacy Applications – that need to be extended and call off-platform services

Typical usage scenarios that use Attunity connect include Application Connectivity to CICS, IMS/TM, Tuxedo, COBOL, and RPG.

(16)

4.3. Adapters

In the Enterprise Application Integration (EAI) space, integration brokers, enterprise service buses (ESB) and business process managers (BPM) provide the backbone for enterprise integration. Yet these tools need to integrate with existing enterprise applications and data sources. Attunity Connect removes the barrier to integrating with legacy applications and data sources by providing standard adapter interfaces and dedicated Plug-Ins to leading adapter frameworks. Adapters include inbound and outbound capabilities, enabling the ability to send messages to the adapter or receive message from the adapters.

Typical applications that require adapters include:

 Integration Brokers – such as BizTalk Server, Oracle BPEL, BEA WLI, etc.

 ESB and BPM platforms

Typical usage scenarios that use Attunity Connect include Application Connectivity to CICS IMS/TM, Tuxedo, COBOL, and RPG, as well as to enterprise data sources.

4.4. Change Data Capture

Data integration projects, especially data warehousing, data synchronization

and propagation projects, need to deal with two main issues: latency and efficiency. On the one hand, users need fresher data with lower latency (i.e. how old the data is), and on the other deal with the efficiencies associated with moving and processing large amounts of data.

Latency, data volumes, and shrinking batch windows are all barriers to data integration.

Attunity Stream removes these barriers by providing an efficient way to processing only the changes to the enterprise data source. Typical applications that require change data capture include:

 ETL - for Data Warehousing and complex data movement

 Data Replication – for re-hosting data (for example – to support reporting)

 Data Synchronization – for maintaining integrity between systems

Typical tools that are used in these scenarios include IBM WebSphere, DataStage, Microsoft SQL Server Integration Services (SSIS), Oracle Warehouse Builder, Business Objects Data Integrator, Synopsis, and others.

(17)

5. ABOUT ATTUNITY

Attunity is a leading provider of real-time event capture and data integration software. Using our software solutions, Attunity’s customers enjoy dramatic business benefits by driving down the cost of managing their operational systems, creating flexible, service-based architectures for increased business agility, and by detecting critical actionable business events, as they happen, for faster business execution.

Attunity has supplied innovative software solutions to its enterprise-class customers for nearly 20 years and has successful deployments at thousands of organizations worldwide.

Attunity provides software directly and indirectly through a number of strategic and OEM agreements with partners such as Microsoft, Oracle, IBM, HP and SAP/Business Objects.

Headquartered in Boston, Attunity serves its customers via offices in North America, Europe, and Asia Pacific and through a network of local partners. For more information, please visit www.attunity.com or email [email protected].

Attunity Integration Suite

Attunity Integration Suite

A White Paper

February 2009

Attunity Integration Suite

Table of Contents

1. INTRODUCTION

2. DATA INTEGRATION – STRATEGIC AND TACTICAL NEEDS

3. ATTUNITY INTEGRATION SUITE

3.1. Attunity Connect

Application Adapters and Service Oriented Integration

Ease of Use

3.2. Attunity Federate

Ease of Use

3.3. Attunity Stream

4. TYPICAL USAGE SCENARIOS

5. ABOUT ATTUNITY