Data Audit Approach to Developing an Enterprise Data Strategy

Alan McSweeney

Data Audit Approach To

Developing An Enterprise

Data Strategy

Objective

• Define a data audit approach to creating an enterprise

current data state view as part of defining an enterprise data strategy

Developing And Implementing An Enterprise Data

Strategy

• Any enterprise data strategy of an existing and mature

organisation with a substantial portfolio of applications and associated data should start with a data audit that establishes a baseline that will be one input to a data strategy

• Any new strategy needs to take into account this (possibly)

substantial applications and data legacy

• Any strategy has to be implementable and operable

• There will be a current state and a future state where the

Current State Desired Long-Term Steady State

Need to Move From Current State To Future

State In A Series Of Steps

Developing And Implementing An Enterprise Data

Strategy

Business Objectives Business Operational Model Enterprise Architecture Solution Implementation and Delivery Management And Operations Business Processes Required Operational Business Systems Business Strategy Systems Design/ Selection Business IT Strategy IT Function Strategy Enterprise Data Strategy Required Operational Processes Required Infrastructure Business Systems Systems Design/ Selection Information and Data Architecture

Enterprise Data Strategy In Context

• An enterprise data strategy exists in a wider organisation and IT

context

− The organisation will have an overall IT strategy to accomplish the organisation strategy and associated objectives

− The IT function will then need its own internal IT strategy that will

structure the function in order to ensure that it can deliver on the wider organisation strategy

− The enterprise data strategy is connected to the overall IT strategy, the enterprise architecture and the internal IT strategy

− The enterprise data strategy will be implemented and operated through an information and data architecture that is part of the overall

enterprise architecture

− This context is important in ensuring that the enterprise data strategy fits into the overall IT and wider organisational structure

− The enterprise data strategy exists to ultimately deliver a business benefit and contribute to the achievement of the business strategy

− The strategy must be translated into an operational framework to enable the strategy to be actualised

Traditional View Of Information And Data

Architecture In An Enterprise Architecture Context

Enterprise Architecture

Data-Oriented View Of Information And Data

Architecture In An Enterprise Architecture Context

Enterprise Architecture

Information and Data Architecture

Information Systems Architecture Solutions and Application Architecture Business Architecture Technology Architecture

Traditional View Of Information And Data

Architecture In An Enterprise Architecture Context

• Data and Information Architecture - the structure of an

organisation's logical and physical data assets and data management resources – is defined as a subset of

Information Systems Architecture which key applications

and data that form the core of mission-critical business processes

• Data and Information Architecture manages the

information of the enterprise by clarifying business relationships and enhancing the understanding of the business processes and rules implemented by the

enterprise

• Data and Information Architecture links Business Processes

It’s All About The Data (And The Processes)

• Data needs to be organised by business process, not by

application

− The enterprise is the sum of its processes

• An effective data architecture is a principal driver of

successful business models and therefore competitive advantage

• Providing business experts timely access to accurate data

is the key factor in improving the ability of the enterprises to make effective and informed business decisions

Components Of An Information And Data

Architecture And Associated Strategy

Information and Data Architecture

Data Governance Data Architecture Management

Data Development Data Operations Management

Data Security Management Data Quality Management

Reference and Master Data Management

Data Warehousing and Business Intelligence Management

Document and Content

Components Of An Information And Data

Architecture And Associated Strategy

• Data Governance - planning, supervision and control over data management and use

• Data Architecture Management - defining the blueprint for managing data assets

• Data Development - analysis, design, implementation, testing, deployment, maintenance

• Data Operations Management - providing support from data acquisition to purging

• Data Security Management - Ensuring privacy, confidentiality and appropriate access

• Data Quality Management - defining, monitoring and improving data quality

• Reference and Master Data Management - managing master versions and replicas

• Data Warehousing and Business Intelligence Management - enabling reporting and

analysis

• Document and Content Management - managing data found outside of databases,

including digital strategy and social media

Information And Data Architecture Components And

Their Functional Elements

• There are a number of

functional elements

associated with each of these components

Data Management Functional Elements

Goals and Principles Activities

Primary Deliverables _{Responsibilities} Roles and

Practices and

Techniques Technology

Organisation and Culture

Information And Data Architecture Components And

Their Functional Elements

• Goals and Principles - directional business goals of each function and the fundamental

principles that guide performance of each function

• Activities - each function is composed of lower level activities, sub-activities, tasks and

steps that are function-specific

• Primary Deliverables - information and physical databases and documents created as

interim and final outputs of each function. Some deliverables are essential, some are generally recommended, and others are optional depending on circumstances

• Roles and Responsibilities - business and IT roles involved in performing and supervising

the function, and the specific responsibilities of each role in that function. Many roles will participate in multiple functions

• Practices and Techniques - common and popular methods and procedures used to perform

the processes and produce the deliverables and may also include common conventions, best practice recommendations, and alternative approaches without elaboration

• Technology - categories of supporting technology such as software tools, standards and

protocols, product selection criteria and learning curves

• Organisation and Culture – this can include issues such as management metrics, critical

success factors, reporting structures, budgeting, resource allocation issues, expectations and attitudes, style, cultural, approach to change management

Why It Happened? Why Is Likely To

Happen In The Future?

What Is Currently Happening?

What Happened?

Every Organisation Aspires To ...

Reporting Insight/

Forecast

Trailing And Leading Indicators

Reporting

• Report on Gathered Information On What Happened

To Understand Pinch Points, Quantify Effectiveness, Measure Resource Usage And Success

Monitoring

• Gather Information In Realtime To Understand

Activities, Respond And Make Reallocation Decisions

Analysis

• Understand Reasons For Outcomes and Modify

Operation To Embed Improvements

Insight and Forecast

• Quantify Propensities, Forecast Likely Outcomes,

Identify Leading Indicators, Create Actionable Intelligence

Trailing Indicators

Leading Indicators

Every Organisation Needs An Effective Enterprise

Data Strategy

Data Operations Management Data Quality Management

Data Development Metadata Management

Document and Content Management Reference and Master Data Management

Data Security Management

Data Warehousing and Business Intelligence Management Data Governance Data Architecture Management Reporting Insight/ Forecast Monitoring Analysis Solid Data Management Foundation and Framework

}

Measurement Framework Iceberg

Be Able To Take Action Based on Reliable Information Measure What is Important Know What Is Important In Order To Measure It Define Measurements Define Consistent Units of Measurements Define Measurement Processes Define Operational Framework Define Collection Process

Define Data Storage Model Define Transformation

And Standardisation Install Data Collection

Facilities Collect Data Monitor Data Collection Manage Data Collection Validate And Store

Data

Report And Analyse Stored Data

Define Reports Run And Distribute

Reports Define Analyses Run And Distribute

Analyses

Provide Realtime Access To Collected

Data Define Data Tools And

Processes Define How The Organisation Delivers Its

Products And Services

Business Function Business Function Business Function Business Function Business Function Partners Regulators Customers Service Providers Suppliers Collaborators

Core And Extended Organisation Landscape

Processes Define How The Organisation Delivers Its

Products And Services

• Work – products and services - moves throughout the

extended organisation landscape as it is delivered to the customer

• Data accompanies – supports, describes, enables,

Cross Functional Processes Crossing “Vertical”

Operational Organisational Units To Deliver Work

Core Cross Functional Processes

• Three cross-functional processes that are common to all

organisations

− Product/service delivery

• From order/specification/design/selection to

delivery/installation/implementation/provision and billing

− Customer management

• From customer acquisition to management to repeat business to up-sell/cross-sell

− New product/service provision

• From research to product/service design to implementation and commercialisation

• These processes cross multiple internal organisation boundaries and

have multiple handoffs but they are what concern customers

• Cross-functional processes deliver value

− Value to the customer

− Value to the enterprise

• Integrated cross-functional processes means better customer service

Core Cross Functional Processes and Customer View

Product/Service Delivery: from order

to completion Customer Relationship Management New Product/ Service Provision

The organisation sees the structure vertically and in a compartmentalised view and all to frequently does not see the customer viewpoint

The customer sees across the structure and is not concerned with but is all too often aware of the operational elements, their complexity and lack of

Organisation Data

• Data flows within the organisation between business

functions, supporting the key processes of:

− Delivery of products and services

− Customer acquisition, management and retention

− Product and service development

• Enterprise data model needs to be structured to define

process interactions and associated data

− Feed data into processes to enable their efficient operation

− Take data from processes to allow their operation to be monitored

Organisation Information And Data Landscape

• Information and data landscape defines the operational

data environment for the organisation

− Operational Use • Storage • Manage • Share • Exchange − Analytic Use • Monitoring • Reporting • Analysis • Forecast

Enterprise Data Model Needs To Encapsulate Data

Landscape

Generalised Enterprise Business Process Model

Business Controlling

Process

Processes That Direct and Tune Other Processes

Core Processes

Processes That Create Value for the Customer

Customer Acquisition Product Delivery Order Fulfilment Customer Support Enabling Processes

Processes That Supply Resources to Other Processes

Channel Management Supply Management Human Resources Information Technology Business Acquisition Business Measurement Process Processes That Monitor and Report the Results of Other Processes Customer’s Process Needs

Supplier’s Processes Business Environment

Generic Enterprise Business Process Model

• Representation of the key processes within and across an

enterprise

− The enterprise is the sum of its processes

• Key processes require and generate data

Data Collection And Measures Need To Be Linked To

Key Enterprise Processes

Business Controlling

Process

Processes That Direct and Tune Other Processes

Core Processes

Processes That Create Value for the Customer

Customer Acquisition Product Delivery Order Fulfilment Customer Support Enabling Processes

Processes That Supply Resources to Other Processes

Channel Management Supply Management Human Resources Information

Technology AcquisitionBusiness

Business Measurement Process Processes That Monitor and Report the Results of Other Processes

Customer’s Process Needs

Supplier’s Processes Business Environment

Competitors, Governments Regulations and Requirements, Standards, Economics

Number of New Customers Customer Turnover Profitability Per Customer Customer Acquisition Cost Number of Customers Complaints Time to Resolve Complaints Delivery Time Accuracy Number of Returns Payment Times Inventory Time to Fulfil Order Invoice Accuracy Forecast Accuracy

Enterprise Data Model Needs To Encapsulate Data

Landscape

Enterprise Data Model

• Build an enterprise data model in layers

• Focus on the most critical business subject areas

− Subject Area Model

− Conceptual Data Model

Subject Area Model

• List of major subject areas that collectively express the

essential scope of the enterprise

• Important to the success of the entire enterprise data

model

• List of enterprise subject areas becomes one of the most

significant organisation classifications

• Acceptable to organisation stakeholders

• Useful as the organising framework for data governance,

Conceptual Data Model

• Conceptual data model defines business entities and their

relationships

• Business entities are the primary organisational structures in a

conceptual data model

• Business needs data about business entities

• Include a glossary containing the business definitions and other

metadata associated with business entities and their relationships

• Assists improved business understanding and reconciliation of terms

and their meanings

• Provide the framework for developing integrated information

systems to support both transactional processing and business intelligence.

Enterprise Logical Data Models

• Logical data model contain a level of detail below the

conceptual data model

• Contain the essential data attributes for each entity

• Essential data attributes are those data attributes without

which the enterprise cannot function – can be a subjective decision

Enterprise Data Model Components

• Data Steward Responsibility Assignments- for subject

areas, entities, attributes, and/or reference data value sets • Valid Reference Data Values - controlled value sets for

codes and/or labels and their business meaning

• Data Quality Specifications - rules for essential data attributes, such as accuracy / precision requirements, currency (timeliness), integrity rules, nullability,

formatting, match/merge rules, and/or audit requirements • Entity Life Cycles - show the different lifecycle states of

the most important entities and the trigger events that change an entity from one state to another

Data Strategy

• High-level course of action to achieve high-level goals

• Data strategy is a data management program strategy a

plan for maintaining and improving data quality, integrity, security and access

• Address all data management functions relevant to the

Elements Of Information And Data Strategy

• Vision for data management

• Summary business case for data management

• Guiding principles, values, and management perspectives

• Mission and long-term directional goals of data management • Management measures of data management success

• Short-term data management programme objectives

• Descriptions of data management roles and business units along

with a summary of their responsibilities and decision rights

• Descriptions of data management programme components and

initiatives

• Outline of the data management implementation roadmap • Scope boundaries

Data Strategy

Data Management Scope Statement

Goals and objectives for a defined planning horizon and

the roles, organisations, and individual leaders accountable

for achieving these objectives

Data Management Programme Charter

Overall vision, business case, goals, guiding principles, measures of success, critical success factors, recognised risks

Data Management Implementation

Roadmap

Identifying specific programs, projects, task assignments, and

Data Audit And Information And Data Strategy

• The objectives of the audit are to understand the current

data management systems, structures and processes

• This will then feed into the development of the strategy

and the identification of gaps

• Data audit views

1. Data landscape view 2. Data supply chain view 3. Data model view

4. Data lifecycle view

5. Current information and data architecture and data strategy view

Data Landscape View

• The purpose of the Data Landscape View is to describe the entities

and functional units within and outside the organisation with which the organisation interacts and to describe the interactions in terms of data flows

• This will show the participants in data flows

• These can be business units, partners, service providers, regulators

and other entities

• The data landscape view can be created at different levels of details:

− Level 1 – Main Interactions - Main interactions and functions associated with

the Enterprise Level

− Level 2 – Business Function - Specific data exchanges of the function

− Level 3 – Function - What is done within each function as a series of activities

− Level 4 – Procedure - How each activity is carried out through a series of tasks

− Level 5 - Sub Procedure - Detailed steps which are carried out to complete a

Data Supply Chain View

• The data supply chain view looks at in-bound and

out-bound data paths within and outside the organisations in terms of the applications and the data that flows along the data paths

• It can be a subset or an extension of the Data Landscape

Data Model View

• Enterprise data model is a set of data specifications that

reflect data requirements and designs and defines the critical data produced and consumed across the

organisation

• Data model view quantifies the status of the development

Enterprise Data Model Needs To Encapsulate Data

Landscape

Data Lifecycle View

• When analysing data, what you are really analysing is the

state of the processes around its lifecycle: how well

defined those processes are, how automated, how risks and controls are defined and managed

Data Lifecycle View

• The stages in this generalised lifecycle are:

− Architect, Budget, Plan, Design and Specify - This relates to the design and specification of the data

storage and management and their supporting processes. This establishes the data management framework

− Implement Underlying Technology- This is concerned with implementing the data-related hardware and

software technology components. This relates to database components, data storage hardware, backup and recovery software, monitoring and control software and other items

− Enter, Create, Acquire, Derive, Update, Integrate, Capture- This stage is where data originated, such as

data entry or data capture and acquired from other systems or sources

− Secure, Store, Replicate and Distribute - In this stage, data is stored with appropriate security and access

controls including data access and update audit. It may be replicated to other applications and distributed

− Present, Report, Analyse, Model - This stage is concerned with the presentation of information, the

generation of reports and analysis and the created of derived information

− Preserve, Protect and Recover- This stage relates to the management of data in terms of backup,

recovery and retention/preservation

− Archive and Recall - This stage is where information that is no longer active but still required in archived

to secondary data storage platforms and from which the information can be recovered if required

− Delete/Remove - The stage is concerned with the deletion of data that cannot or does not need to be

retained any longer

− Define, Design, Implement, Measure, Manage, Monitor, Control, Staff, Train and Administer, Standards, Governance, Fund - This is not a single stage but a set of processes and procedures that cross all stages

and is concerned with ensuring that the processes associated with each of the lifestyle stages are

Data Audit Approach

1. Build an application landscape view, including internal and external systems and third-parties from which data may be obtained and to which data may be supplied

− The application view can be supplement with a system and infrastructure view that shows the hardware and software components behind an application

2. Layer onto this information capture, storage and flows: where and what types of information is maintained by applications and that is passed between applications

− An application is a collection of systems and infrastructure that delivers an integrated set of functions

− It may or may not be necessary to document the underlying infrastructure associated with applications

− This may be further complicated because the underlying infrastructure may not be isolated but may itself be part of an application - this would be the case where the server infrastructure is virtualised and managed by

virtualisation manager

3. Categorise information by a classification such as: Operational Data, Master and Reference Data, Analytic Data and Unstructured Data

4. Define the business units/functions and their use of applications

5. View the information capture, storage and flows identified above across the stages of their lifecycle

6. Identify how well the processes and their controls associated with the lifecycle stages are defined, documented and operated. This will identify gaps to be remediated

Data Audit Approach – Application Landscape

Data Audit Approach – Data Capture, Storage And

Transfer

Data Audit Approach – Infrastructure And System

View

Database Server Database Server

Load Balancer Load Balancer Authentication Server

User Directory

Firewall Firewall

Consists of

Classification Information By Operational Data, Master

and Reference Data, Analytic Data and Unstructured Data

Architect, Budget, Plan, Design and Specify

Enter, Create, Acquire, Derive, Update, Integrate, Capture

Secure, Store, Replicate and Distribute

Preserve, Protect and Recover Archive and Recall

Delete/Remove

Implement Underlying Technology

Present, Report, Analyse, Model

Define, Design, Implement, Measure, Manage, Monitor, Control, Staff, Train and Administer,

Standards, Governance, Fund

Operational Data Analytic and Derived Data Unstructured Data Master and Reference Data

Business Functions And Application Use

Application 1 Application 2 Application 3

Application 4 Application 5 Application 6

Application 7 Application 8 Application 9

Business Function 1 Business Function 2 Business Function 3 Business Function 4

Information Capture, Storage And Flows Identified

Above Across The Stages Of Their Lifecycle

Architect, Budget, Plan, Design and Specify

Enter, Create, Acquire, Derive, Update, Integrate, Capture

Secure, Store, Replicate and Distribute

Preserve, Protect and Recover Archive and Recall

Delete/Remove

Implement Underlying Technology

Present, Report, Analyse, Model

Define, Design, Implement, Measure, Manage, Monitor, Control, Staff, Train and Administer,

Standards, Governance, Fund

Data Type 1 Data Type 3 Data Type 4 Data Type 2

Identify How Well The Processes And Their Controls

Associated With The Lifecycle Stages Are Defined

Architect, Budget, Plan, Design and Specify

Enter, Create, Acquire, Derive, Update, Integrate, Capture

Secure, Store, Replicate and Distribute

Preserve, Protect and Recover Archive and Recall

Delete/Remove

Implement Underlying Technology

Present, Report, Analyse, Model

Define, Design, Implement, Measure, Manage, Monitor, Control, Staff, Train and Administer,

Standards, Governance, Fund

Data Type 1 Data Type 3 Data Type 4 Data Type 2

Identify How Well The Processes And Their Controls

Associated With The Lifecycle Stages Are Defined

• Provides a baseline of the status of data processes in the

organisation

• Identify gaps to be remediated

• This will then form the basis of a workplan to resolve any

Current Information and Data Architecture And Data

Strategy and View

• Review current information and data architecture and implementation and operational under the key

component areas

Information and Data Architecture

Data Governance Data Architecture Management

Data Development Data Operations _Management

Data Security Management Data Quality Management

Reference and Master Data Management

Data Warehousing and Business Intelligence

Management

Document and Content

Current Data Management View

• The data strategy components and the functional

elements are be combined to create a view of all the potential elements of an operational data strategy implementation and operational framework

• Not all of these facets will have the same importance

• Each of these facets will also be in a different state of

effective operation

• You can create a high-level representation of the state of

Data Management View – Components And

Functional Elements

Management  Scope of Each Data Management Function 

Data Security Management Data Quality Management Reference and Master Data Management Data Warehousing and Business Intelligence Management Document and Content Management Metadata Management

Goals and Principles Activities Primary Deliverables Roles and Responsibilities Practices and Techniques Technology Organisation and Culture

Importance Current _State Importance Current _State Importance Current _State Importance Current _State Importance Current _State Importance Current _State Importance Current _State

Data Governance Data Architecture Management Data Development Data Operations Management Data Security Management Data Quality Management Reference and Master

Data Management Data Warehousing and

Business Intelligence Management Document and Content Management Metadata Management

= High Importance = Medium Importance = Low Importance = Good State = Medium State = Poor State

Data Management View – Importance and Status

• Coding of data management components and functional

elements

• Understand their importance and current state of

Data Audit Views And Results

• Data Landscape View – quantify and understand where data exists • Data Supply Chain View – quantify and understand data exchanges

and interfaces

• Data Model View – quantify and understand the development and

specification of the enterprise data model

• Data Lifecycle View – identify how well the processes and the

controls associated with the lifecycle stages are defined

• Current Information And Data Architecture And Data Strategy View

– identify current information and data architecture and

implementation and operational under the key component areas

• Current Data Management View – quantify the relative importance

and current state of implementation and operation of data management components and functional elements

Data Audit Views And Results

• Gives a comprehensive view of the current state, desired

future state and gaps/deficiencies

• Provides a current state view within the context of a future

state

• Ensures that any information and data architecture and

strategy is based on evidence

• Enables a realistic workplan to be developed and worked

through to achieve the desired results

• Approach can be applied to the entire enterprise or

Now All That Is Left Is The Implementation And

Operation

More Information

Alan McSweeney