42% of companies blame multiple databases for their data quality issues. 75% of companies waste an average of 14% of revenue due to bad data quality

(1)

Better BI through Master Data

Management

Jeremy Pritchard

Photo Credit : NASA

42% of companies blame

multiple databases for

their data quality issues

Experian Data Quality Survey

Photo Credit : Tim Dobbelaere

75% of companies

waste an average of

14% of revenue due to

bad data quality

Experian Data Quality Survey

Photo Credit : Howard Lake

Master Data Management

improves business performance

Photo Credit : AP

What is Master Data?

Master data is the consistent and

uniform set of identifiers and

extended attributes that describes

the core entities of the enterprise

including customers, prospects,

citizens, suppliers, sites,

hierarchies and chart of accounts.

Gartner

Master data is data that is shared by

multiple computer systems.

The Information Difference

Master data is information

that is key to the operation

of a business…persistent,

non-transactional data that

defines a business entity for

which there is, or should be,

an agreed-upon view across

the organisation.

Wikipedia

Master data is often one of the

key assets of a company.

Microsoft

What is Master Data Management?

Master data management is a

technology-enabled discipline in

which business and IT work

together to ensure the uniformity,

accuracy, stewardship, semantic

consistency and accountability of

the enterprise’s official shared

master data assets.

Gartner

Master Data Management

comprises a set of processes,

governance, policies,

standards and tools that

consistently defines and

manages the master data

.

Wikipedia

The creation of:

The Golden Record Single Version of the Truth

(2)

Types of data in an organisation

The What, Why, and How of Master Data Management – Microsoft November 2006

Master

Hierarchical

Transactional

Unstructured

Metadata

• The master data elements are the nouns and are people, things,

and places

• The transactional data elements are verbs that describe what

happens to those people, places, and things.

Understanding Master Data

• Think of nouns and verbs

• Bob Smith buys a widget (SKU #A1234) and ships it to his home address

CRM Marketing ERP WMS Financial

widget (SKU #A1234)

Bob Smith

home address

Deciding what Master Data should be Managed

Reuse

Value

Volatility

Cardinality

Lifetime

Data Quality Improvement Concept

Data Governance Share Communicate Analyse Propagate Data Distribution Build Match Merge Data Mastering Improve Standardise Enrich Data Quality Know Explore Profile Data Analysis Get Connect Orchestrate Data Integration Manage Control

Data Quality Improvement Concept

Data Governance Communicate Analyse Propagate Data Distribution Build Match Merge Data Mastering Improve Standardise Enrich Data Quality

Data Governance



It embodies:



Data quality



Data management



Data policies



Business process management



Risk management



Data governance is a quality control discipline for:



Assessing



Managing



Using



Improving

People

Process

(3)

Data Integration - Batch

Data Integration – Real Time

Profiling



Basic Analysis



Patterns / Masking



Extremes



Quantities



Frequency Analysis

Profiling

Monitoring

•

Advanced Profiling

•

‘Custom’ analysis of data

•

Defined by user and relevant to data context

•

Output is Binary (true/false) – Data Quality Indicators

(4)

Data Governance - Monitoring

Portal

Reports Profiling/DQIs

DQ plan

Data Quality Improvement Concept

Cleansing

•

Parsing

– Data parsed into components (pattern based) E.G. Jim Smith -> Jim + Smith

•

Validation

– Validation of Data Quality against rules – Validation of Data Quality against reference tables

•

Enrichment

– Adding data

•

Standardisation

– Transformation into standard format (16-Feb-75 > 16/02/1975) – Standard and nonstandard abbreviations (Str. -> Street) – Language-specific replacements Standardisation Enrichment Validation Parsing Cleansing

Scoring

Standardisation Enrichment Validation Parsing Cleansing

Data Before and After Cleansing

Name ANNE PHILLIPS CHRISTINE HALL JOHN SMITH IAN SCOTT

Gender F N Male

Date of Birth 14/11/1987 10/12/1940 10/01/1971 28.Oct.1956 Telephone 01569 274873 01491 24778 01598 867305 7801551340 Email [email protected] [email protected] [email protected] ian@@dfgmail.-.com Address Line 1 6 BOOTON COURT 56C HORNCHURCH ROAD 22 RINGMORE STREET 56 WOULD LANE Address Line 2

Address Line 3

Address Line 4 KIDDERMINSTER PLYMUTH ISLEWORTH Address Line 5 PORCESTERSHIRE DEVON LONDON MIDDLESEX Postcode DY102YZ PL5 2TF SE233DE TW7-5ED

Score 0 210 300 600

Explanation ADDRESS_VALID GENDER_TAKEN_FROM_NAME ADDRESS_CORRECTED_MINOR GENDER_TAKEN_FROM_NAME ADDRESS_CORRECTED_MINOR EMAIL_INV DATE_STANDARDIZED GENDER_STANDARDIZED TELEPHONE_STANDARDIZED ADDRESS_CORRECTED_MAJOR

out_first_name Anne Christine John Ian out_last_name Phillips Hall Smith Scott

Name JOHN SMITH out_first_name John

out_last_name Smith

Gender out_gender M

Date of Birth 10/01/1971 out_birthdate 10/01/1971 Telephone 01598 867305 out_telephone 01598 867305 Email [email protected] out_email [email protected] Address Line 1 22 RINGMORE STREET out_address_line_1 22 RINGMORE RISE Address Line 2 out_address_line_2

Address Line 3 out_address_line_3 Address Line 4 out_address_line_4 Address Line 5 LONDON out_post_town LONDON

Name IAN SCOTT out_first_name Ian

out_last_name Scott

Gender Male out_gender M

Date of Birth 28.Oct.1956 out_birthdate 28/10/1956 Telephone 7801551340 out_telephone 07801 551340 Email ian@@dfgmail.-.com out_email

Address Line 1 56 WOULD LANE out_address_line_1 56 WOOD LANE Address Line 2 out_address_line_2

Address Line 3 out_address_line_3 Address Line 4 ISLEWORTH out_address_line_4 Address Line 5 MIDDLESEX out_post_town ISLEWORTH

Data Governance – Issue Resolution

Yes

Is the score lower than

the threshold?

(5)

Portal

Reports Profiling/DQIs

DQ plan

Data Governance - Issue Management

Workflow Issue List Issue data Issue Database Exception Mgt

Master Data Management

Name: Bob Smith Tel: 01323-456842 DOB: Gender: Male

Name: Smith, Bob Tel: (01283)56982 DOB: 23/10/1971 Gender: Name: B Smith Tel: (0)1323456842 DOB: 23-Oct-71 Gender: M Name: Bob Smith Tel: 01323 456842 DOB: Gender: M Name: B Smith Tel: 01323 456842 DOB: 23/10/71 Gender: M

Name: Bob Smith Tel: 01283 56982 DOB: 23/10/71 Gender: Name: Bob Smith

Tel: 01323 456842 DOB: 23/10/71 Gender: M

CRM Marketing ERP WMS Financial

Master Data Management Architectures

Consolidated

• Master is Single Version of Truth

• Data Quality at Master • Updates occur at Sources • Updates propagated to

Master

Coexistence

• Data Quality is ongoing • Updates occur at Sources or

Master

• Updates propagated to other Sources

Registry

• Multiple Versions of Truth • Data Quality is ongoing • Updates occur at Sources • Keys and Metadata in

Registry • Updates optionally

propagated to other Sources

Centralised

• Data Quality at Master • Updates occur at Master • Updates propagated to Sources

+

Matching

Goal:

Identify groups of records that in reality

represent a single client or entity.

Match & Merge

How many people are here?

Cleansed data

First

Last

G SIN

Birth Date

Address

John Smith M 16/12/1978 22 Ringmore Rise, London, SE23 3DE

John Smith M 095242434 16/12/1978 22 Ringmore Rise, London, SE23 3DE

John Smith M 095242434 74 Arnold Street, Boldon Colliery, Bolton, NE35 9BD

Smith M 16/11/1978

John Smith M 095252433 16/11/1978 3 Catalina Avenue, Pembroke Dock, SA72 6YB

John Smith M 16/11/1978 3 Catalina Avenue, Pembroke Dock, SA72 6YB

John Smiht M 095252433 16/11/1978

Jane Watson F 420347213 3 Catalina Avenue, Pembroke Dock, SA72 6YB

Jane Watson F 420347213 01/01/1982 3 Catalina Avenue, Pembroke Dock, SA72 6YB

Jane Smith F 420347213 01/01/1982

(6)

Cleansed data

First

Last

G SIN

Birth Date

Address

Smith M 16/11/1978

John Smiht M 095252433 16/11/1978

Jane Smith F 420347213 01/01/1982

J. Smith 420347213

Match

Merging



Creating the Golden Record



Can cherry pick the best fields or even the best record



For example:



The one from the ‘reference system’



The newest one



The one of highest quality

Match & Merge

Cleansed data

First

Last

G SIN

Birth Date

Address

Smith M 16/11/1978

John Smiht M 095252433 16/11/1978

Jane Smith F 420347213 01/01/1982

J. Smith 420347213

Match

Golden record

First

Last

G

SIN

Birth Date

Address

Cleansed data

First

Last

G SIN

Birth Date

Address

Merge

John Smith M

095242434 16/12/1978

74 Arnold Street, Boldon Colliery, Bolton, NE35 9BD

The newest permanent address The most frequent

address 22 Ringmore Rise, London, SE23 3DE

Data Governance Communicate Analyse Propagate Data Distribution Build Match Merge Data Mastering Improve Standardise Enrich Data Quality

Data Distribution

(7)

Intelligence

Operational vs. Analytical

Master Data Management

Operational vs. Analytical Master Data Management

Operational MDM centres on assuring ‘single view’ of master data

in the core systems used by business users

Sales, service, order management, manufacturing, purchasing, billing, accounts

receivable, accounts payable, payroll, etc.

Rely heavily on integration technologies to keep systems in sync

Operational

Operational data is fundamental to the running of an organisation

Operational MDM

• Global Cruise Company

• Customer held in multiple systems…

 Customer data can be entered in all systems (except Marketing)

 No real-time checking of existing customer in other systems

 No proper link between customers in systems

 Business performance is impacted:

 Customer service reps having to use two systems (legacy and CRM) to deal

with customers

 Passport information captured in Ship System but not shared for future use

 Problems with loyalty scheme

 Duplicates fed into Marketing system – customers being marketed to multiple

times

Booking Legacy CRM Marketing Ship

Operational vs. Analytical Master Data Management

Analytical MDM centres on assuring ‘single view’ of master data

in the downstream data warehouse used most often to supply the data

for a business intelligence (BI) solution for historical and predictive analysis

Any data cleansing done inside an Analytical MDM solution is invisible to the

transactional applications

Analytical

Analytical data is used to support a company's decision making

Analytical MDM

 Global accountancy training company

 Sell to individuals not companies

 Have no idea how much companies are spending

with them

 Would like to be able to build stronger relationships

with organisations

(8)

Operational

Analytical

Single Version of Truth = Better

 System synchronisation  Consistency in transactional data  Party/product data across all systems  System integration/migration  Cost reduction within the business

process

 Data aggregation & analysis  Marketing segmentation & analysis  Risk management

 Financial reporting  Cost reduction and time savings in

analysis

Maximum business value comes from managing both operational and analytical master data

Master Data Management - Value Across the Enterprise

The Reality

Adoption and Plans for MDM

Information Difference – 2013

Reported Success of MDM Programs

Information Difference – 2013

The Right Way to Implement Your

MDM Program

• Executive Sponsor

• Business Case