Adabas Options for Unlocking Your Data

(1)

Optimize Manage Extend

Adabas Options for

Unlocking Your Data

Mark Braukhoff

(2)

2

10-Jan-20

Natural

Conference

Challenges To ...

Integrating Data Sources and Applications

Mainframe

Midrange

PC

Databas e Databas e Databas e Databas e

Applicatio

n

Applicatio

n

Applicatio

n

Applicatio

n

Applicatio

n

Applicatio

n

Applicatio

n

Databas e Databas e

■

Multiple Platforms & OS

■

Multiple Databases & Apps

■

Heavy workload

■

Many parallel users

■

Availability around the clock

■

Data and Web integration

■

Real-time data changes

(3)

3

10-Jan-20

Natural

Conference

New Adabas offerings help you...

Integrate Adabas Data Sources Across Your Enterprise

Adabas SQL Gateway

Reactively Pulls Data

Proactively Pushes Data

(4)

Optimize Manage Extend

Introducing

Adabas Replication

(5)

5

10-Jan-20

Natural

Conference

Market Demand

Customers are telling us they need to ……

Improve their methods of data integration to support more

time sensitive business intelligence and operational needs.

(6)

6

10-Jan-20

Natural

Conference

The Software AG Difference

More than just technology…

Software -

foundation to provide capabilities to solve your data replication needs,

Expertise -

to configure the software per your specific requirements, and

Support -

of

your specific solution via our award-winning customer support center

(7)

7

10-Jan-20

Natural

Conference

Adabas Replication Solution

■

It Helps You To:

-

Optimize real-time performance

-

Eliminate costly work-arounds

-

Improve responsiveness to your

internal and external customers

Adabas Replication Solution enables real-time data propagation of

(8)

8

10-Jan-20

Natural

Conference

Adabas Replication Solution

Architecture & Capabilities

Data Capture & Processing

• Immediate Access to COMMITTED Transactions (ET) • Meta Data oriented for subscriber

• Deals with all transaction types (insert, update, delete)

• User defined recovery process • Asynchronous Processing • Near Real-Time

• Supports multi-update nucleus environments

Special Transaction Handling

• Back out processing

• Transaction consolidation for efficiency • Maintain transaction chronology

• File decompression

• User specified transformation & filtering for the recipient system

Fine-grained “Change Data Capture”

processing

• User defined subscription based transaction replication

• User-definable to field level for target • Initial State (Bi-directional)

• Replication Audit at Subscription level to ensure system health

(9)

9

10-Jan-20

Natural

Conference

Adabas Replication Solution Delivery

Implement Pilot

▪ Installation of

Replication Framework

▪ Configuration one

replication flow

Deploy to

Production

▪ Testing

▪ Scaling & Optimization

▪ Production Transition

▪ Training & Knowledge

Transfer

Discover & Assess

▪ Understand the

customer need

▪ Define Business Problem

▪ Define Requirements

▪ Define the vision for the

solution

▪ Create Business Case

▪ Create Solution

Blueprint (System

Architecture &

Implementation Plan)

Support

▪ 24X7 Support

▪ Problem Resolution

▪ Enhancement Request

Knowledge Transfer

Assess

Construct

Deploy

Support

(10)

10

10-Jan-20

Natural

Conference

Boston University

Customer Example

Supporting “On-Demand” Student Requests

Student

Information

System (SIS)

Degree Audit

Repository

Unix

Intra/Internet

4

3

2

1

1. Student Requests Degree Information

2. Degree Audit Application Requests Data from SIS System

3. SIS application delivers data to Degree Audit Application

4. Degree Audit Application delivers information to

student(s)

* Slow response time with high-volume demand.

Student

Registration

Financial Aid

and Payments

Housing & Meal

Plans

(11)

11

10-Jan-20

Natural

Conference

Student

Information

System

(Natural / Adabas)

Mainframe

Intra/Internet

1

3

2 Adabas Replication

1. Adabas Replication allows the question - “Who has had their student records changed?” -

in real time.

2. Business Rules (Natural), independent from student requests, allows the infrastructure to

activate the updating of Degree Audit records in a timely fashion.

3. When a student requests the re-building of a Degree Audit package “on-demand” (24x7)

from Degree Audit Application, the need to require a new Degree Audit package from the

mainframe is no longer mandatory.

* Goal: Excellent performance without maintaining additional 3

rd

party software.

Boston University

Customer Example

Supporting “On-Demand” Student Requests

Degree Audit

Repository

(Java/DB2)

(12)

12

10-Jan-20

Natural

Conference

Increased demand for

more current and

accurate information from

transactional systems

Maintaining a

home-grown solution to

accomplish “change data

capture” or “event

processing”

Improve Customer Service

And

Reduce Costs Across Multiple

Touch Points

Eliminate Risks

And

Reduce Operational Costs

Do you need Adabas Replication Solution?

Business Drivers

IT

Delivers

Business

(13)

13

10-Jan-20

Natural

Conference

Potential Scenarios

1. Disaster Recovery

1. Physical Disasters

2. “Dude, where’s my data?”

2. Information Reporting

3. Information Publication

4. Information Distribution

(14)

14

10-Jan-20

Natural

Conference

Disaster Recovery

-

Replication of Adabas data-sets

from place to place, selectively or in entirety

Example:

improve recoverability, availability & maintenance

Administration

ASSO DATA WORK

Replicated Data Initial-State Requests Transformed Replicated Data Transformed Messages Initial-State Requests Adabas Replication Buffer Adabas Nucleus Data Replication Server Data Replication Buffer Messaging System

Database and Recovery Data

ASSO DATA WORK

(15)

15

10-Jan-20

Natural

Conference

Information Reporting -

Supply data to a single

target, which may be combined with data from other

sources

Example:

populate a data-warehouse/data-mart to support

business reporting/business analysis

Adabas

Systems

Other

Transactional

Systems

Data

Marts

Replicated/Transformed Adabas Data

(16)

16

10-Jan-20

Natural

Conference

Adabas

Systems

Other

Transactional

Systems

Operational Systems

Replicated/Transformed Adabas Data

Other Data

Target Application

Information Publication

to Support Operational

Systems - Supply of Adabas data to subscribing

applications/data stores

Example:

a message-oriented intermediary - managing subscriber

access and data delivery - to multiple independent operational systems.

Messaging System

(17)

17

10-Jan-20

Natural

Conference

Information Distribution

-

Replication of a single Adabas source to multiple

targets, making multiple local copies of centralized data.

Example:

mobile computing applications or distributed business

intelligence applications

Adabas

Systems

Other

Transactional

Systems

Web Services

Replicated Data

XML

Transformed Data

(18)

18

10-Jan-20

Natural

Conference

Adabas SQL Gateway

Reactively Pulls

Data

Proactively Pushes

Data

Adabas Replication Solution

Extend your Adabas data to applications and systems

across the enterprise, and…

• Are integrated into the entire Software AG product line

• Fully compatible with all Adabas products

• Fully supported through Customer Support Center

(19)

19

10-Jan-20

Natural

Conference

Adabas

Replication Buffer

Adabas

Nucleus

Messaging

System

Target

Application

Replicator

Replication Buffer

Transformed

Messages

Replicated

Data

Adabas Replication Facility

Data Replication: Technical Overview

Adabas

Replication

(20)

20

10-Jan-20

Natural

Conference

Data Replication: Technical Overview

Adabas Replication Definitions

-

Define

Replication for the Database (GCB)

• Identify default Replicator Server ID

-

Define Files requiring Replication (FCB)

• Identify the Primary Key (optional)

• Filter of Modification Types (Store, Update, Delete)

• Identify requirements for the replication data

(Before/After/DVT information)

• Identify Replicator Server ID

-

Session assignment of a Replication Buffer

(21)

21

10-Jan-20

Natural

Conference

Data Replication: Technical Overview

Replicator Definitions

-

Create Replicator repository

-

Create Subscription Definition

• Assign Destination(s) queues

• Definition through AOS subsystem

V8 - Gui application through Adabas System Manger

• Rules to Select Data (commands, files)

• Rules for extracting and formatting columns (field

• Customized Rules to be applied to replication data

User Exit option

-

Session assignment of a Replication Buffer

(22)

22

10-Jan-20

Natural

Conference

Data Replication: Technical Overview

Replicator Definitions

-

Subscription Rule Definitions

• Rules to Select Data

Filter by Command type; Store, Delete, Update

Filter by Files number

Filter using Verification Rules

-

Change in Field value

-

Field value satisfies defined Boolean logic criteria

• Rules for extracting and formatting columns

Format Buffer

Before/After Image criteria

Encoding rules

• Predict Integration

(23)

23

10-Jan-20

Natural

Conference

19:00:20 A D A B A S REPTOR Parameter Subsystem 2004-05-20 Subscription Definition M-RP1410

Description ... SUBSCRIPTION 01_________________

Subscription Name ... SUBS0001 Current User Data Alpha Key ... 1___

Architecture Key ... 4___ Subscription Version ... 1_ User Data Wide Key ... 4___

Destination Name List ... _ File Related Parameters ... _

Command ==>

Help Quit Save Exit

Adabas Replication Facility

(24)

24

10-Jan-20

Natural

Conference

Adabas Replication Facility

19:01:00 A D A B A S REPTOR Parameter Subsystem 2004-05-20 File Related Parameters M-RP1420

Description: SUBSCRIPTION 01 Subscription Name ... SUBS0001 Current

DBID ... _____ File Number ... ____ Replicate for Insert ... _ Replicate for Update ... _ Replicate for Delete ... _

Subscription User Exit ... ________

Before Image FB ... _ -or- Before Image GFB Name ... ________ After Image FB ... _ -or- After Image GFB Name .... ________ Key Related Before Image FB _ -or- Key Relater GFB Name .... ________

Command ==>

Help Quit Save Exit

(25)

25

10-Jan-20

Natural

Conference

Data Replication: Technical Overview

Replicator Definitions continued

-

Create Destination Definitions

• Queue Name(s)

• Target Type (e.g. Adabas, Mediator, Tamino)

• Communication Protocol

Messaging Tool (e.g. Communicator, MQ)

Native TCP/IP

Natural RPC

XML Pipe

SOAP Messaging

• Deployment Rules (Payload size)

• Support Redundancy for through-put and error conditions.

• Support Dynamic destination re-assignment.

(26)

26

10-Jan-20

Natural

Conference

18:57:19 A D A B A S REPTOR Parameter Subsystem 2004-05-20 ADABAS Destination Definition M-RP1235

Destination Subscription Input Input Target Target Name Name Dbid File Dbid File --- ________ ________ _____ ____ _____ ____ _____ ____ _____ ____ _____ ____ _____ ____ _____ ____ _____ ____ _____ ____ _____ ____ _____ ____ _____ ____ _____ ____ _____ ____ _____ ____ _____ ____ _____ ____ _____ ____ _____ ____ _____ ____

Command ==>

Help Quit Add Save - + Exit

Adabas Replication Facility

(27)

27

10-Jan-20

Natural

Conference

18:56:15 A D A B A S REPTOR Parameter Subsystem 2004-05-20 ET-Broker Destination Definition M-RP1220

Dest Name ... ________ Entire Broker ID .. ________________________________ Architecture ... __ Broker Service .... ________________________________ Commit Threshold .. ____ Service Name ... ________________________________ Service Class ... ________________________________

Command ==>

Help Quit Save Exit

Adabas Replication Facility

(28)

28

10-Jan-20

Natural

Conference

18:56:01 A D A B A S REPTOR Parameter Subsystem 2004-05-20 MQSeries Destination Definition M-RP1210

Destination Name .... ________ Architecture ... __ Threshold ... ____ Queue Manager Name... ________________________________________________ Queue Name ... ________________________________________________

Command ==>

Help Quit Save Exit

Adabas Replication Facility

(29)

29

10-Jan-20

Natural

Conference

Data Replication: Technical Overview

Replicator Definitions continued

-

Define Recovery Levels

• Persistence of Replication Data

• Audit Trail of all Replicated Transactions per subscriptions

• ‘System under Stress’ protocol

-

Define ‘Feedback’ queues

• Replay request (timestamp oriented)

• Refresh request (refresh data)

• Re-do request (resend replicated transaction)

(30)

30

10-Jan-20

Natural

Conference

Data Replication: Processing Flow

■

Command Processing

-

Replication initiated by

Insert, Delete, Update commands

-

ET commands trigger Replication Event

-

BT commands suppress Replication Event

■

Data Collection

-

Replication initiated only for first modification to a replicated file

-

Records collected by ISN per File per Transaction

-

Replicated Data Integrity managed via Hold Queue logic

-

Maintain Before/After Images in Replication Buffer

-

Maintain a directory of all modified data per transaction in

Replication Buffer

(31)

31

10-Jan-20

Natural

Conference

Data Replication: Processing Flow

■

Data Delivery to Replicator

-

Data shipped as part of ET; after WORK/PLOG done

-

Serialization of change enforced using a FIFO Queue

-

Elimination of redundant data (one instance of an ISN per file)

■

Replication Data Selection

-

Scan Subscriptions for DBnr/Fnr, command type match

-

Apply verification rules of selection

-

Extended Rule verification i.e. customization logic

-

Decompression of data per Format definition

(32)

32

10-Jan-20

Natural

Conference

Data Replication: Processing Flow

■

Data Distribution

-

Asynchronous delivery via Messaging Tool (e.g. Communicator)

-

Delivery to Target Application

-

Recovery and persistence considerations

-

Multiple Destination support

(33)

33

10-Jan-20

Natural

Conference

Admin

Adabas

Replication Buffer

Adabas

Nucleus

Messaging

System

Target

Application

Replicator

Replication

Buffer

Adabas Replication Facility

Adabas

Replication

Server

ASSO DATA

WORK

Recovery

Data

Replicator Repository

TLOG

Data Replication: Processing Flow

(34)

34

10-Jan-20

Natural

Conference

Reptor

Server

Adabas Replication Facility

DB20

Nucid 85

Adabas

DB 20

Nucid 50

Adabas

DB 10

Adabas

DB 20

Nucid 15

Target

Adabas

z/OS

Image B

Unix

Server D

z/OS

Image C

z/OS Image A

Data Replication: Processing Flow in a Cluster

Sysplex

CF

(35)

35

10-Jan-20

Natural

Conference

Data Replication Recovery

Adabas

-

Adabas Replication Buffer

• Unconfirmed Replication Transactions to Replictor

-

Adabas Transactions Persistent on WORK

• Integrated with Adabas Auto-Restart

■

Replication Server

-

Maintain Audit Trail of all Replicated Transactions

-

‘System under Stress’ protocol

• Suspend and drain queues for certain transactions

• Recovery and Resume normal replication

-

Persistent Subscription Log of actual Replicated Transactions

• Restart after loss in application connectivity

(36)

36

10-Jan-20

Natural

Conference

Data Replication Recovery

Application Recovery

-

Regenerate Replicated transactions from sequential PLOG

-

Refresh request

• Refresh data for an ISN or ISN Range

• Refresh data for a specific key value

• Refresh data for a key value range

• Refresh data for a file

-

Re-send request for Replicated Transaction

■

Operator option to suspend replication for a

Subscription or file

(37)

37

10-Jan-20

Natural

Conference

Adabas

Replication Buffer

Adabas

Nucleus

Messaging

System

Reptor

Replication

Buffer

Adabas Replication Facility

Adabas

Replication

Server

Reptor Repository

TLOG

Data Replication Recovery

SLOG

Subscription

Transaction Log

WORK

Transaction Data

Replicated Transaction Data

_Transaction

(38)

38

10-Jan-20

Natural

Conference

Data Replication Summary

Integrated Solution

-

Support by Adabas through internal enhancements

-

Support for other SAG products

-

Support for certain non SAG products

■

Performance centric

■

Minimal Impact on normal nucleus activity

■

Near Real Time Support

■

Support for Distributed Transaction Processing across

tiers and heterogeneous software components

■

Flexible and extendible

■

Strategic for Enterprise Data Sharing

■

Strategic for Disaster Recovery and Event capture

(39)

39

10-Jan-20

Natural

Conference

New Adabas offerings help you...

Integrate Adabas Data Sources Across Your Enterprise

Adabas SQL Gateway

Reactively Pulls Data

Proactively Pushes Data

Adabas Replication

Solution