SAP BI

(1)

SAP BI - Data Quality with Business

Objects Data Services

SAP NetWeaver BI taps into Data Services

(2)

1. Motivation

1.1. Data Quality as an issues

2. Business Objects Data Services in Detail

2.1. Introduction

2.2. Features and Functions

2.3. Which product for which requirement

3. The SAP BI <-> Data Services Use Cases

4. I - Data Quality Made Easy

3.1. Easy to consume Data Service features

3.2. Include Data into SAP NetWeaver BI staging

5. II - Data Quality for Experts

4.1. Address / Data Cleansing

4.2. Matching

4.3. Auditing

6. III – Closed Loop Scenario

7. Summary

(3)

Do You Trust Your Information?

Up to

75%

have made

wrong business

decisions due to

flawed data

Only

10%

always have the information they

need to make business decisions

They spend up to

30%

of their time verifying

the accuracy and quality of the data they

use to make decisions

*Survey of information workers in the US, Great Britain, France, and

Germany, commissioned by Business Objects and conducted by Harris

Interactive

June 2006

Survey of information workers*

(4)

Business

IT

Information Gap

Need timely access to

trusted data

Changing business

requirements

Making decisions with

knowledge shadows

Limited capacity

to support users

Competing priorities

Lengthy ETL and data

quality development

cycles

Is your IT organization able to keep

up with information demands?

(5)

Data Quality Challenges for

Data Warehousing

Data Warehouse Challenges in Data Quality

General

Incorporation of multiple sources of data

Completeness of data

Credibility of the data

Tracking of origin of data (including data lineage)

Define strategy based on source data

Master data

Value check (including plausibility, ranges, etc.)

Structure of data (Pattern)

Standardizing of the data

Elimination of duplicate records

Transactional data Datenqualität

Referential integrity

Checksums on key figures

Value check (including plausibility,

thresholds, lookups, etc.)

(6)

1. Motivation

1.1. Data Quality as an issues

2. Business Objects Data Services in Detail

2.1. Introduction

2.2. Features and Functions

2.3. Which product for which requirement

3. The SAP BI <-> Data Services Use Cases

4. I - Data Quality Made Easy

3.1. Easy to consume Data Service features

3.2. Include Data into SAP NetWeaver BI staging

5. II - Data Quality for Experts

4.1. Address / Data Cleansing

4.2. Matching

4.3. Auditing

6. III – Closed Loop Scenario

7. Summary

(7)

Business Objects is a market leader

in data quality

Gartner Magic Quadrant for Data

Quality Tools, 2008

Source: Gartner Sept., 2008

Market Leader

(8)

ERP

DW

RDBMS

OLAP

XML

Docs

Web

Email

Notes

Structured Data

Unstructured Data

Data

Integration

Data

Integration

Data Quality

Data Profiling

MDM

ERP, SRM, CRM

Applications

Performance

Management

Business

Intelligence

Metadata

Management

Metadata

Management

Information Lifecycle

Management

Information Lifecycle

Management

Enterprise Information Management

Data Layer

SAP Business Objects –

A Comprehensive solution for EIM

(9)

SAP Data Services

Runtime

Architecture

Metadata

Repository

Development

User Interface

Administration and Connectors

Runtime

Architecture

Metadata

Repository

Development

User Interface

Administration and Connectors

Data Integrator XI R2

Data Quality XI R2

Data Services is the first single tool for data integration and data quality

One Runtime

Architecture

One Development UI

One Metadata Repository

One Administration Environment

Profile

Transform

Deliver

Access

Cleanse

Data Services XI 3.0

(10)

One Development User Interface

for Data Integration and Quality

Integrate

heterogeneous data

across the enterprise

Profile and cleanse

any type of data anywhere in the enterprise

Integrate

heterogeneous data

across the enterprise

(11)

SAP ERP, SAP CRM,

SAP MDM,

SAP NetWeaver BI,

…

Shared Metadata

Impact Analysis

Data Lineage

Data prof

iling

Data Services Architecture

Data

Services

Engine

Data

Auditing

Data

Validation

Data

Cleansing

Real

Time

Batch

Files, XML,

Mainframe,

Excel, etc.

Oracle, SQL,

DB2, etc.

PSFT,

Oracle Apps,

Siebel, etc.

R/3, ERP,

NetWeaver BI

Query,

Reporting,

Analysis

& Dashboards

Data Migration,

Synchronisation, …

SOA

(12)

1. Motivation

1.1. Data Quality as an issues

2. Business Objects Data Services in Detail

2.1. Introduction

2.2. Features and Functions

2.3. Which product for which requirement

3. The SAP BI <-> Data Services Use Cases

4. I - Data Quality Made Easy

3.1. Easy to consume Data Service features

3.2. Include Data into SAP NetWeaver BI staging

5. II - Data Quality for Experts

4.1. Address / Data Cleansing

4.2. Matching

4.3. Auditing

6. III – Closed Loop Scenario

7. Summary

(13)

Profile And Cleanse

Complete and global data quality

Measure and analyze data through

data assessment and continuous

monitoring

Cleanse and enhance customer

and operational data anywhere

across the enterprise

Match and consolidate data at

multiple levels within a single pass

for individuals, households, or

corporations

Improve and automate the delivery

of

direct mail and goods

Data Quality Framework

(14)

1. Motivation

1.1. Data Quality as an issues

2. Business Objects Data Services in Detail

2.1. Introduction

2.2. Features and Functions

2.3. Which product for which requirement

3. The SAP BI <-> Data Services Use Cases

4. I - Data Quality Made Easy

3.1. Easy to consume Data Service features

3.2. Include Data into SAP NetWeaver BI staging

5. II - Data Quality for Experts

4.1. Address / Data Cleansing

4.2. Matching

4.3. Auditing

6. III – Closed Loop Scenario

7. Summary

(15)

Side-by-Side comparison

Requirement

Referential Integrity

Plausibility Check

Pattern matching

Lookups

BI Master Data attribute lookup

Profiling

Address cleansing

Data Cleansing

Matching

Formula support

Custom routines and functions

Auditing

(16)

1. Motivation

1.1. Data Quality as an issues

2. Business Objects Data Services in Detail

2.1. Introduction

2.2. Features and Functions

2.3. Which product for which requirement

3. The SAP BI <-> Data Services Use Cases

4. I - Data Quality Made Easy

3.1. Easy to consume Data Service features

3.2. Include Data into SAP NetWeaver BI staging

5. II - Data Quality for Experts

4.1. Address / Data Cleansing

4.2. Matching

4.3. Auditing

6. III – Closed Loop Scenario

7. Summary

(17)

Data Services for SAP NetWeaver BI

- Use Cases

Data Services complements the intrinsic BI capabilities

Apply Data Quality measures to Non-SAP data

Cleanse SAP BI data

Call Data Services

from BI staging

(not scope of this

presentation)

SAP NetWeaver BI

Any

Source

SAP

WebService UDC

Address

Cleansing

1

3

2

1

2

3

3 Use case

(18)

1. Motivation

1.1. Data Quality as an issues

2. Business Objects Data Services in Detail

2.1. Introduction

2.2. Features and Functions

2.3. Which product for which requirement

3. The SAP BI <-> Data Services Use Cases

4. I - Data Quality Made Easy

3.1. Easy to consume Data Service features

3.2. Include Data into SAP NetWeaver BI staging

5. II - Data Quality for Experts

4.1. Address / Data Cleansing

4.2. Matching

4.3. Auditing

6. III – Closed Loop Scenario

7. Summary

(19)

Scenario I:

Simple Data Quality by Data Services

Data Services XI 3.0 – Basic Checks

Profiling of source data

Domain value and plausibility check

Pattern Matching

String Matching

…

Data Services XI 3.0 – Basic Checks

Profiling of source data

Domain value and plausibility check

Pattern Matching

String Matching

…

SAP BI Staging

1

1 Scenario

(20)

Data Profiling in Data Services

Understand your data

-Need to understand the data before creating an ETL process

Check for missing values (NULL)

Get possible list of values

Visualize the data distribution

Find patterns

Get data ranges (min, max, average) – identify data domain outliers

Uniqueness of data (distinct values)

Can also be used to:

Verify results of an ETL load during development

Analyze data for system migrations

Loading additional data such as potential leads or purchased lists

(21)

Example of Profiling

(22)

Data Validation – check your data

Use either Validation or Query Transform

Check data in respect to

Domain / plausibility check

Validity checks

Ranges checks (for dates, postal codes, etc.)

Examine Data Structures based on patterns for

Phone Numbers

Dates & Times

General Numbers

Use Boolean expressions and custom coding for complex requirements

Find records by

Search strings

Wildcard search

(23)

Data Validation in Data Services

(24)

1. Motivation

1.1. Data Quality as an issues

2. Business Objects Data Services in Detail

2.1. Introduction

2.2. Features and Functions

2.3. Which product for which requirement

3. The SAP BI <-> Data Services Use Cases

4. I - Data Quality Made Easy

3.1. Easy to consume Data Service features

3.2. Include Data into SAP NetWeaver BI staging

5. II - Data Quality for Experts

4.1. Address / Data Cleansing

4.2. Matching

4.3. Auditing

6. III – Closed Loop Scenario

7. Summary

(25)

Incorporate Data in SAP BI staging

Motivation

Benefit from Data Services tools and features

Easy access to Non-SAP sources

Use scheduling feature from SAP BI

Trigger process by SAP BI

Pre-requisites:

Based on RFC call

Create RFC destination in SAP BI

Connect SAP BI and Data Services

Data Services as Source System in SAP BI

SAP BI as target or source in Object Library of Data Services

Start RFC Server for Data Services

(26)

Steps for Incorporation of Data

Define BI

in Data

Services

Object

Library

Import

InfoSource

Metadata

into Data

Services

Define

Info-Package

to

schedule

execution

of Data

Services

Job

Incorpo-rate

Info-Source in

Data

Services

Job

Export

Job

execution

(for batch

execution

by SAP

BI)

Check

Result in

PSA or

respec-tive data

target

Define

Data

Services

as Source

System in

SAP BI

(27)

1. Motivation

1.1. Data Quality as an issues

2. Business Objects Data Services in Detail

2.1. Introduction

2.2. Features and Functions

2.3. Which product for which requirement

3. The SAP BI <-> Data Services Use Cases

4. I - Data Quality Made Easy

3.1. Easy to consume Data Service features

3.2. Include Data into SAP NetWeaver BI staging

5. II - Data Quality for Experts

4.1. Address / Data Cleansing

4.2. Matching

4.3. Auditing

6. III – Closed Loop Scenario

7. Summary

(28)

Scenario II:

Data Quality for Experts

Data Services XI 3.0 – Complex Checks

Address / Data Cleansing

Matching

Auditing

…

Data Services XI 3.0 – Complex Checks

Address / Data Cleansing

Matching

Auditing

…

SAP BI Staging

1

1 Use case

(29)

Data Cleansing

Cleanses and standardizes party data such as

names/addresses, emails, phone numbers,

SSNs, and dates

Manages international data for over 200

countries and reads and writes Unicode data

Removes errors to uncover true content of

database

Improves integrity of data to identify matches

and ultimately create a single customer view

Parses and standardizes non-party data

Such as account numbers, product codes,

product descriptions, purchase dates, part

numbers, SKUs, etc.

Utilizes a rule-based parsing and rule editing

architecture for even greater customized results

(30)

Maggie.kline@future_electronics.com

Margaret Smith-Kline phd

FUTURE Electronics

5/23/03

101 6th ave

manhattan

ny

10012

001124367

Data Cleansing (Person record)

Salutation:

Ms.

First name:

Margaret

Last name:

Smith-Kline

Post name:

Ph. D.

Match standards: Maggie, Peg, Peggy

Gender:

Strong Female

Company name: Future

Electronics

Address 1:

101 Avenue of the Americas

City:

New York

State:

NY

ZIP+4:

10013-1933

Email:

maggie.kline@

future_electronics.com

SSN:

001-12-4367

Date:

May 23, 2003

Input record

Output record

(31)

Data Cleansing (Product Data)

Description

Kallkyle screw

test steel plate 20 x 35 mm

wire 23.33 x 40.50 cm

34 x 60 mm steel plate

steel plate 34,0 60 mm

34.0 x 60,0 mm steel plate

34 x 60 mm steel plate ?

plate

steel plate

Input

Parsed output

Product

Dimension

Type

Form

screw

_Kallkyle

plate

20x35 mm

_steel

_test

wire

23.33 x 40.50 cm

plate

34 x 60 mm

_steel

plate

34 x 60 mm

_steel

plate

34 x 60 mm

_steel

plate

34 X 60 mm

_steel

plate

_steel

(32)

1. Motivation

1.1. Data Quality as an issues

2. Business Objects Data Services in Detail

2.1. Introduction

2.2. Features and Functions

2.3. Which product for which requirement

3. The SAP BI <-> Data Services Use Cases

4. I - Data Quality Made Easy

3.1. Easy to consume Data Service features

3.2. Include Data into SAP NetWeaver BI staging

5. II - Data Quality for Experts

4.1. Address / Data Cleansing

4.2. Matching

4.3. Auditing

6. III – Closed Loop Scenario

7. Summary

(33)

House-holding data to identify members of same household,

corporation or any other hierarchy

Identifying “snowbirds”

i.e. individuals or households with multiple residences

Creating a panoramic single best record

Preventing firms from doing business with entities on

government watch lists, Do-Not-Mail, prison lists, etc…

Providing identity resolution to uncover non-obvious

relationships for fraud detection

Matching and Consolidation

Unlocking the relationships between distinctly different sets

of data

(34)

Matching and Consolidation

Ms Margaret Smith-Kline Ph.D.

Future Electronics

101 Avenue of the Americas

New York NY 10013-1933

maggie.kline@future_electronics.com

May 23, 2003

Name:

Ms. Margaret

Smith-Kline Ph.D.

Company name:

Future Electronics Co. LLC

SSN:

001-12-4367

Purchase date:

5/23/2003

Address:

101 Avenue of the Americas

City:

New York, NY 10013-1933

Latitude:

40.722970

Longitude:

-74.005035

Fed code:

36061

Phone:

(222) 922-9922

Email:

maggie.kline@

future_electronics.com

Input records

Consolidated record

Maggie Smith

Future Electronics Co. LLC

101 6th Ave.

Manhattan, NY 10012

maggie.kline@future_electronics.com

001-12-4367

Ms. Peg Kline

Future Elect. Co.

101 6th Ave.

New York NY 10013

001-12-4367

(222) 922-9922

5/23/03

(35)

1. Motivation

1.1. Data Quality as an issues

2. Business Objects Data Services in Detail

2.1. Introduction

2.2. Features and Functions

2.3. Which product for which requirement

3. The SAP BI <-> Data Services Use Cases

4. I - Data Quality Made Easy

3.1. Easy to consume Data Service features

3.2. Include Data into SAP NetWeaver BI staging

5. II - Data Quality for Experts

4.1. Address / Data Cleansing

4.2. Matching

4.3. Auditing

6. III – Closed Loop Scenario

7. Summary

(36)

Data Auditing

Audit the quality of the ETL process itself

Compare data

Before (source)

During (transformations) and

After the ETL process (target)

Set audit points to calculate audit values

Count records

Calculate Checksums, Sum and Averages for numeric columns

Define audit rules based on Boolean Expressions

Raise alerts or notification via Email when these rules are violated.

(37)

Data Auditing in Data Services

Audit rule: Check the number of records at the beginning and End of Data

Quality process

$Count_SAP_CUSTOMER_DATA = $Count_SAP_CUSTOMERS_CLEANSED)

Audit action: Send message to administrator and / or

write entry in error log

(38)

1. Motivation

1.1. Data Quality as an issues

2. Business Objects Data Services in Detail

2.1. Introduction

2.2. Features and Functions

2.3. Which product for which requirement

3. The SAP BI <-> Data Services Use Cases

4. I - Data Quality Made Easy

3.1. Easy to consume Data Service features

3.2. Include Data into SAP NetWeaver BI staging

5. II - Data Quality for Experts

4.1. Address / Data Cleansing

4.2. Matching

4.3. Auditing

6. III – Closed Loop Scenario

7. Summary

(39)

Scenario III:

SAP BI and Data Services – Closed-loop

Data Services XI 3.0 - Address Cleansing

SAP BI customer address data in BI

Hand over SAP BI customer addresses to Data

Services using existing OpenHub API

Load cleansed data back to SAP BI

…

Data Services XI 3.0 - Address Cleansing

SAP BI customer address data in BI

Hand over SAP BI customer addresses to Data

Services using existing OpenHub API

Load cleansed data back to SAP BI

…

SAP BI Open Hub

Service

SAP BI Staging

2

2 Use case

(40)

1. Motivation

1.1. Data Quality as an issues

2. Business Objects Data Services in Detail

2.1. Introduction

2.2. Features and Functions

2.3. Which product for which requirement

3. The SAP BI <-> Data Services Use Cases

4. I - Data Quality Made Easy

3.1. Easy to consume Data Service features

3.2. Include Data into SAP NetWeaver BI staging

5. II - Data Quality for Experts

4.1. Address / Data Cleansing

4.2. Matching

4.3. Auditing

6. III – Closed Loop Scenario

7. Summary

(41)

Integration &

Data Quality

SAP BI &

Data

Services

_Delivered

Dictionaries

Data Quality

all-around

Extend the SAP BI

capabilities

Use country-specific

information for address

cleansing and matching

From simple validation to complex

cleansing and matching operations

Key Points to Take Home

Access to Non-SAP sources

enriched with Data Quality

in one tool

(42)

Further Information

SAP Public Web:

SAP Developer Network (SDN):

www.sdn.sap.com

SDN Business Object area:

https://www.sdn.sap.com/irj/boc

Blog

https://weblogs.sdn.sap.com/pub/wlg/12040

HowTo Guides (

https://www.sdn.sap.com/irj/sdn/howtoguides

)

How To Use Data Services I - Data Quality Made Easy

https://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/b0e611

92-b296-2b10-ca90-a21eea43f569

How To Use Data Services II - Data Quality For Experts

https://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/c0f79d

98-b396-2b10-9098-db6b2890d190

(43)

(44)

No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG. The information contained herein may be changed without prior notice.

Some software products marketed by SAP AG and its distributors contain proprietary software components of other software vendors.

SAP, R/3, xApps, xApp, SAP NetWeaver, Duet, SAP Business ByDesign, ByDesign, PartnerEdge and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries all over the world. All other product and service names mentioned and associated logos displayed are the trademarks of their respective companies. Data contained in this document serves informational purposes only. National product specifications may vary.

The information in this document is proprietary to SAP. This document is a preliminary version and not subject to your license agreement or any other agreement with SAP. This document contains only intended strategies, developments, and functionalities of the SAP® product and is not intended to be binding upon SAP to any particular course of business, product strategy, and/or development. SAP assumes no responsibility for errors or omissions in this document. SAP does not warrant the accuracy or completeness of the information, text, graphics, links, or other items contained within this material. This document is provided without a warranty of any kind, either express or implied, including but not limited to the implied warranties of

merchantability, fitness for a particular purpose, or non-infringement.

SAP shall have no liability for damages of any kind including without limitation direct, special, indirect, or consequential damages that may result from the use of these materials. This limitation shall not apply in cases of intent or gross negligence.

The statutory liability for personal injury and defective products is not affected. SAP has no control over the information that you may access through the use of hot links contained in these materials and does not endorse your use of third-party Web pages nor provide any warranty whatsoever relating to third-party Web pages

Weitergabe und Vervielfältigung dieser Publikation oder von Teilen daraus sind, zu welchem Zweck und in welcher Form auch immer, ohne die ausdrückliche schriftliche Genehmigung durch SAP AG nicht gestattet. In dieser Publikation enthaltene Informationen können ohne vorherige Ankündigung geändert werden.

Einige von der SAP AG und deren Vertriebspartnern vertriebene Softwareprodukte können Softwarekomponenten umfassen, die Eigentum anderer Softwarehersteller sind. SAP, R/3, xApps, xApp, SAP NetWeaver, Duet, SAP Business ByDesign, ByDesign, PartnerEdge und andere in diesem Dokument erwähnte SAP-Produkte und Services sowie die dazugehörigen Logos sind Marken oder eingetragene Marken der SAP AG in Deutschland und in mehreren anderen Ländern weltweit. Alle anderen in diesem Dokument erwähnten Namen von Produkten und Services sowie die damit verbundenen Firmenlogos sind Marken der jeweiligen Unternehmen. Die Angaben im Text sind unverbindlich und dienen lediglich zu

Informationszwecken. Produkte können länderspezifische Unterschiede aufweisen.

Die in diesem Dokument enthaltenen Informationen sind Eigentum von SAP. Dieses Dokument ist eine Vorabversion und unterliegt nicht Ihrer Lizenzvereinbarung oder einer anderen Vereinbarung mit SAP. Dieses Dokument enthält nur vorgesehene Strategien, Entwicklungen und Funktionen des SAP®-Produkts und ist für SAP nicht bindend, einen bestimmten Geschäftsweg, eine Produktstrategie bzw. -entwicklung einzuschlagen. SAP übernimmt keine Verantwortung für Fehler oder Auslassungen in diesen Materialien. SAP garantiert nicht die Richtigkeit oder Vollständigkeit der Informationen, Texte, Grafiken, Links oder anderer in diesen Materialien enthaltenen Elemente. Diese Publikation wird ohne jegliche Gewähr, weder ausdrücklich noch stillschweigend, bereitgestellt. Dies gilt u. a., aber nicht ausschließlich, hinsichtlich der Gewährleistung der Marktgängigkeit und der Eignung für einen bestimmten Zweck sowie für die Gewährleistung der Nichtverletzung geltenden Rechts.

SAP übernimmt keine Haftung für Schäden jeglicher Art, einschließlich und ohne Einschränkung für direkte, spezielle, indirekte oder Folgeschäden im Zusammenhang mit der Verwendung dieser Unterlagen. Diese Einschränkung gilt nicht bei Vorsatz oder grober Fahrlässigkeit.

Die gesetzliche Haftung bei Personenschäden oder die Produkthaftung bleibt unberührt. Die Informationen, auf die Sie möglicherweise über die in diesem Material enthaltenen Hotlinks zugreifen, unterliegen nicht dem Einfluss von SAP, und SAP unterstützt nicht die Nutzung von Internetseiten Dritter durch Sie und gibt keinerlei Gewährleistungen oder Zusagen über Internetseiten Dritter ab.