Mastering Data Management. Mark Cheaney Regional Sales Manager, DataFlux

(1)

Mastering Data Management

Mark Cheaney

Regional Sales Manager,

DataFlux

(2)

Today, the amount of technical information

(3)

(4)

There are over 31 Billion searches on Google every month

LOADING

(5)

In 2006, this number was 2.7 Billion

(6)

(7)

(8)

Times Are Changing…



1 out of 4 workers have been in their job less than one year. 1 out of 2 less than five years

(9)

Are We Keeping Up?



Over 80 US banks failed in 2009



The US government has taken majority ownership of General Motors, Freddie Mac, Fannie Mae, AIG…



Société Générale lost $7.5B in a 2008 derivatives trading fiasco



Valparaiso, Indiana had an $8M budget shortfall in 2007

(10)

(11)

Mastering Data

Management

(12)

Is Data Managed Across Your Enterprise?

Mastering Data Management

Is Data a Trusted Business Asset?

(13)

Sales Force Automation Database Marketing IT-driven projects Duplicate, inconsistent data Inability to adapt to business changes Data Warehouse ERP CRM Line of business influences IT projects Little cross-functional collaboration High cost to maintain

multiple applications IT and business groups collaborate Enterprise view of certain domains Data is a corporate asset Customer MDM Product MDM Business requirements drive IT projects Repeatable, automated business processes Personalized customer relationships and optimized operations MDM Business Process Automation

(14)

(15)

How Do We Master Data?

Establish the people and policies for

data governance

Focus data management on business process

improvement

Standardize on a data management

technology platform

(16)

Data Governance –

People and Policies

(17)

IT Business

Data Governance: IT and Business Collaboration

Executive Sponsorship

Data Governance

Council

Data Steering

(business experts)

Data Management

Data Administration Data Architecture

Security and Privacy

LOB Data Governance Data Stewards

(18)

58%

No

83 No

Management Support Collaboration

Data Governance – Executive Support

Little to No Support Noticeable or Better Support Little Collaboration Collaboration

(19)

Originally published in “A Data Governance Manifesto” by Jill Dyché. Used with permission from Baseline Consulting.

Accountable Consulted Informed

Data Governance – Regimes

Sales Customer

Service Finance Marketing

Human Resources Data Governance Council Procurement Campaign Management Hiring Order Management Billing

Trouble Ticket Tracking

Core Business Processes

(20)

Data Governance – Policy



Creation, documentation (including business vocabulary), approval

process and maintenance of data standards for form, function, meaning and versioning



Quality and stewardship for data elements, business rules, hierarchies, taxonomies and content tagging



Creation and maintenance of enterprise data model and enterprise data services

(21)

Business Process

Improvement

(22)

(23)

Traditional Data Management Approach

Data Source Data Source Data Source

Data Rule Data Rule Data Rule Data Rule Data Rule Data Rule

Data Rule Data Rule Data Rule

Data Domain

(24)

Emerging Data Management Approach –

Mastering Data for Business

Business Domain

Data Source

Data Rule Data Rule

Data Rule

Data Source

Data Rule

Data Source

Data Rule

Trusted, Integrated Data

Business Policy

Business Info Business Info Business Info

Business Policy

(25)

Data Management

Platform

(26)

DataFlux UnityPlatform

(27)

Reporting and Dashboards

Design and Development Environment Business Vocabulary/Data Definitions

Data Access

Business Rule and Event Processing and Monitoring Data Archiving

Data Privacy and Security Metadata Management

(28)

Identity Resolution

Business Rule Creation and Management

Data Enrichment Metadata Discovery

Hierarchy and Reference Data Definition Unstructured Data Discovery

Verification, Normalization, Standardization, Transformation Data Exploration and Profiling

(29)

Data Services and SOA Data Synchronization

ETL/ELT

Business Process Integration Merging and Clustering Business Rules Execution

Grid Computing

(30)

Business Data Services

Domain Data Models

Master Data History/Auditing and Exception Reporting Entity Definition/Management and Search

(31)

How Do We Master Data?

Establish the people and policies

for data governance

Focus data management on business

process improvement

Standardize on a data management

technology platform

(32)

(33)

(34)

(35)

(36)

Data Profiling

Identify data quality issues

Determine if data fits requirements Identify business process issues

(37)

Real-Life Profiling Exercises

A financial services company knew of 3 genders: M, F, and blank. They did not know about X and C.

A home care products company discovered shipments slated for

16’x16’ pallets. The IS manager wondered what kind of truck they would go on.

Prior to a VA audit, a cross-check of medical billings by a healthcare provider showed it was performing open heart surgeries in

ambulances.

Consumer products mfr. learned a product of theirs was railroad boxcars.

(38)

Analyze - Profiling

Table, Column, & Relationship Metrics

Pattern Recognition Visualization

(39)

Data Profiling

Uncover Problematic or Inconsistent Data

View detailed information on the accuracy,

completeness, consistency, structure, uniqueness and validity of data

Create and share reports to build consensus on data quality and data

(40)

Data Quality

Correct identified data quality issues Normalize inconsistent data

(41)

Data content

Missing & Invalid data. Data domain outliers.

Illogical combinations of data

Data structure and

storage

Uniqueness

Referential integrity

Migration/integration

Normalization inconsistencies. Duplicate or lost data

Standards

Ambiguous Business Rules

Multiple Formats for Same Data Elements

Different Meanings for the Same Code Value.

Multiple Codes Values with the Same Meaning

Field Overuse: used for unintended purpose.

Data in Filler

(42)

(43)

Data Integration

Identify and eliminate duplicates Identify and link households

(44)

Data Integration

SFA ERP

Data Warehouse Call Center

Apex Equipment | Pittsburgh

PA Apex LLC | Pittsburgh, Penn

Apex Construction | Pittsburgh PA Apex Equip & Const | Pitt PA

Apex Equipment & Construction, LLC | Pittsburgh PA 15233

Data Integration Data Quality

Data Model Business Services Stewardship Console Business User Interface

Data Governance Identity Management

Reporting Data Profiling Metadata Discovery Business Rule Definition

(45)

Data Enrichment

Make data more useful

Add postal information to improve customer outreach

Append product codes to speed procurement and materials management efforts

(46)

Data Enrichment

Validate and verify

Data validation and verification ensures data accuracy

Test data against other data sources (internal or external) known to be correct or current

Product code verification (industry-standard codes, UPC, ISDN) Address verification (ZIP codes, geocoding)

Validated data Input

940 Cary Pkw Cary NC

27503

940 NW Cary Pkwy Ste 201

Cary NC

27513-4355 County: Wake Census Tract: 452.2

(47)

Data Enrichment

(48)

Data Enrichment

(49)

Data Monitoring

Data integrity checks & balances.

Business rule development by business analysts.

(50)

Data Monitoring

Maintain High-Quality Data Over Time

Ensure clean data stays clean

Validate data against your business rules Automatically identify invalid data

(51)

About DataFlux

Recognized as a leading provider of data quality, data integration, and MDM solutions

Provides a unique single platform to analyze, improve and control enterprise data

Over 1,200 customers worldwide

Offices in the US, the UK, France and Germany Founded in 1997

Acquired by SAS, the world’s largest privately owned

software company, in 2000

(52)

Questions

DataFlux Midwest Manager Mark Cheaney

mark.cheaney@dataflux.com

630-799-8058

For more information, visit: