The First Step in Information Management
Big Data and Big Data Governance
Kelle O’Neal
kelle@firstsanfranciscopartners.com
415-‐425-‐9661
4
Table of Contents
Big Data Value and Impact
Enterprise InformaKon Management
Big Data Management
Big Data Governance
4
4
Big Data Importance
When integrated with other enterprise data, organizaKons can
develop more insighTul understanding of their business which
can lead to:
A stronger compeKKve edge
Improve business processes
Greater product innovaKon and improvements
Increase in growth and revenue
Increased employee producKvity through streamlined business
processes
4
ImplicaKons of Big Data
How will organizaKons have to be designed, organized, and managed?
What exisKng business models are likely to be disrupted?
How will organizaKons’ legacy business models and technology
compete?
How will business processes change?
How will markeKng funcKons and acKviKes have to evolve?
How will organizaKons leverage and value their data assets?
How will execuKves help their organizaKons take advantage of the
change that is under way?
Where do they start and how?
Current technologies and data management structures in organiza3ons no longer work
in this new era of big data
4
4
A Comprehensive Framework
Provides a holisKc view of data in order to manage data as a corporate asset
Enterprise InformaKon Management
InformaKon Strategy
Architecture and Technology Enablement
Content Delivery
Business Intelligence
and Performance Management
GOVERNANCE
ORGANIZATIONAL ALIGNMENT
Content Management
4
Develop and execute architectures, policies and procedures to manage the full data lifecycle
How Big Data Fits
Enterprise Data Management
Ensure data is available, accurate, complete and secure
Data Quality
Management
Data Architecture
RetenKon/Archiving
Data
Master Data
Management
Reference Data
Management
Metadata Management
Management
Big Data
Privacy/Security
4
4
FoundaKon to Harness Internal and External Data
BDM provides foundational capabilities to integrate and analyze data from non-traditional data
sources in order to find insights in new types of data
Process
Automation
Architectural
Improvements
Flexible Data
Architecture
IT
Transformation
and
Adaptability
PAST
PRESENT
FUTURE
Transaction
Management
Data
Warehousing
Master Data
Management
Integrated Information
Management and Delivery
Process automation and management of transactions with application specific data
within isolated business applications including ERP, CRM, SCM, eCommerce and
other systems over the past decade
Data extraction and normalization for operational as well as management
reporting and functional analytics. Data integrity and lack of standards have
constrained the maturity of analytics in the past
MDM is management of foundational data domains that support core
business processes, information and insight creation. It provides for
flexibility data integration, directly supporting enterprise
information architecture vision
EIM and adaptive architecture to
deliver business capabilities and
flexibility to future changes
Big Data
Management
BDM is integraKng and managing big data and its relaKonship
across the enterprise through people, processes and
technology. It provides opportunity to find insights in new
types of data and content, to make organizaKons more agile,
and to answer quesKons that were previously considered
beyond reach
4
Big Data Lifecycle Process
Listen
Capture
Process
Integrate
Analyze
Consume
4
Types of Data
Data Disciplines are expanding. Most types of data are not
completely independent. Big Data o7en has a rela9onship to
other data types. Management of these data sets addresses:
•
Data Quality
•
Enrichment /Enhancement
•
Relevance
•
Privacy and Security
•
Governance
Small Data
Big Data
Reference Data
Master Data
Metadata
4
Data Types Work Together
Master Data
Enhanced/Enriched
Master Data
(360 degree View)
Examples:
•
Social Media Influence
•
Social Media Account IDs
•
Demographic InformaKon
•
RelaKonships
•
Email IDs
•
Validated Master Data
Examples include:
•
Address validaKon
through loca9on
broadcasts and geo-‐
loca9on data
Big Data (Interaction Data)
Big or Small Data
(Transactional Data)
Reference Data
4
4
Data Governance DefiniKon
Data Governance is the organizing
framework for establishing strategy,
objecKves and policy for effecKvely
managing corporate data.
It consists of the processes, policies,
organizaKon and technologies required to
manage and ensure the availability,
usability, integrity, consistency, audit
ability and security of your data.
CommunicaKon
and Metrics
Data
Strategy
Data Policies and
Processes
Data
Standards
and
Modeling
A Data Governance Program consists of the
inter-workings of strategy, standards,
4
• Vision & Mission • Objectives & Goals • Alignment with Corporate
Objectives
• Alignment with Business Strategy
• Guiding Principles
• Statistics and Analysis • Tracking of progress • Monitoring of issues • Continuous Improvement • Score-carding
• Policies & Rules • Processes • Controls
• Data Standards & Definitions • Metadata, Taxonomy,
Cataloging, and Classification • Operating Model
• Arbiters & Escalation points • Data Governance
Organization Members • Roles and Responsibilities • Data Ownership &
Accountability
• Collaboration & Information Life Cycle Tools
• Data Mastering & Sharing • Data Architecture & Security • Data Quality & Stewardship
Workflow • Metadata Repository • Communication Plan • Mass Communication • Individual Updates • Mechanisms • Training Strategy • Business Impact & Readiness
• IT Operations & Readiness • Training & Awareness
• Stakeholder Management & Communication • Defining Ownership & Accountability
Change
Management
4
CompeKng PrioriKes
Business
Insight
Security
& Control
4
• Vision & Mission • Objectives & Goals • Alignment with Corporate
Objectives
• Alignment with Business Strategy
• Guiding Principles
• Statistics and Analysis • Tracking of progress • Monitoring of issues • Continuous Improvement • Score-carding
• Policies & Rules • Processes • Controls
• Data Standards & Definitions • Metadata, Taxonomy,
Cataloging, and Classification • Operating Model
• Arbiters & Escalation points • Data Governance
Organization Members • Roles and Responsibilities • Data Ownership &
Accountability
• Collaboration & Information Life Cycle Tools
• Data Mastering & Sharing • Data Architecture & Security • Data Quality & Stewardship
Workflow • Metadata Repository • Communication Plan • Mass Communication • Individual Updates • Mechanisms • Training Strategy • Business Impact & Readiness
• IT Operations & Readiness • Training & Awareness
• Stakeholder Management & Communication • Defining Ownership & Accountability
Change
Management
Strategy
•
Extension of overall Data Governance
Strategy and Scope
•
Business purpose and value unique of Big
Data
•
Understanding of impacted business
processes and key requirements
4
• Vision & Mission • Objectives & Goals • Alignment with Corporate
Objectives
• Alignment with Business Strategy
• Guiding Principles
• Statistics and Analysis • Tracking of progress • Monitoring of issues • Continuous Improvement • Score-carding
• Policies & Rules • Processes • Controls
• Data Standards & Definitions • Metadata, Taxonomy,
Cataloging, and Classification • Operating Model
• Arbiters & Escalation points • Data Governance
Organization Members • Roles and Responsibilities • Data Ownership &
Accountability
• Collaboration & Information Life Cycle Tools
• Data Mastering & Sharing • Data Architecture & Security • Data Quality & Stewardship
Workflow • Metadata Repository • Communication Plan • Mass Communication • Individual Updates • Mechanisms • Training Strategy • Business Impact & Readiness
• IT Operations & Readiness • Training & Awareness
• Stakeholder Management & Communication • Defining Ownership & Accountability
Change
Management
OrganizaKon
•
New Stakeholders
•
Extended parKcipaKon at all levels to
include Privacy, new Lines of Business
•
Extended RACI to cover new data types
•
Redefine role and scope of Data Steward;
idenKfy new stewards
•
New roles (i.e. Data ScienKsts)
•
New Regions
4
• Vision & Mission • Objectives & Goals • Alignment with Corporate
Objectives
• Alignment with Business Strategy
• Guiding Principles
• Statistics and Analysis • Tracking of progress • Monitoring of issues • Continuous Improvement • Score-carding
• Policies & Rules • Processes • Controls
• Data Standards & Definitions • Metadata, Taxonomy,
Cataloging, and Classification • Operating Model
• Arbiters & Escalation points • Data Governance
Organization Members • Roles and Responsibilities • Data Ownership &
Accountability
• Collaboration & Information Life Cycle Tools
• Data Mastering & Sharing • Data Architecture & Security • Data Quality & Stewardship
Workflow • Metadata Repository • Communication Plan • Mass Communication • Individual Updates • Mechanisms • Training Strategy • Business Impact & Readiness
• IT Operations & Readiness • Training & Awareness
• Stakeholder Management & Communication • Defining Ownership & Accountability
Change
Management
Policies, Processes & Standards
•
Extension of Security, Privacy Policies
•
Policies around data masking in tesKng
and/or producKon, and “unmasking”
•
Understanding of Intellectual Property
consideraKons and Appropriate Use
•
Extension of Data RetenKon Policy
Archiving
Storage
DisposiKon
•
Policy Enforcement
•
Metadata, ClassificaKon
•
New DefiniKons and Terms
4
• Vision & Mission • Objectives & Goals • Alignment with Corporate
Objectives
• Alignment with Business Strategy
• Guiding Principles
• Statistics and Analysis • Tracking of progress • Monitoring of issues • Continuous Improvement • Score-carding
• Policies & Rules • Processes • Controls
• Data Standards & Definitions • Metadata, Taxonomy,
Cataloging, and Classification • Operating Model
• Arbiters & Escalation points • Data Governance
Organization Members • Roles and Responsibilities • Data Ownership &
Accountability
• Collaboration & Information Life Cycle Tools
• Data Mastering & Sharing • Data Architecture & Security • Data Quality & Stewardship
Workflow • Metadata Repository • Communication Plan • Mass Communication • Individual Updates • Mechanisms • Training Strategy • Business Impact & Readiness
• IT Operations & Readiness • Training & Awareness
• Stakeholder Management & Communication • Defining Ownership & Accountability
Change
Management
Measurement & Monitoring
•
Re-‐evaluate Data Quality
Standards, Thresholds and
Metrics
•
Data Availability
requirements and
monitoring
•
Data Profiling rules &
processes
•
Monitoring of data
movement and usage
•
Track security, privacy
•
Web metrics
4
• Vision & Mission • Objectives & Goals • Alignment with Corporate
Objectives
• Alignment with Business Strategy
• Guiding Principles
• Statistics and Analysis • Tracking of progress • Monitoring of issues • Continuous Improvement • Score-carding
• Policies & Rules • Processes • Controls
• Data Standards & Definitions • Metadata, Taxonomy,
Cataloging, and Classification • Operating Model
• Arbiters & Escalation points • Data Governance
Organization Members • Roles and Responsibilities • Data Ownership &
Accountability
• Collaboration & Information Life Cycle Tools
• Data Mastering & Sharing • Data Architecture & Security • Data Quality & Stewardship
Workflow • Metadata Repository • Communication Plan • Mass Communication • Individual Updates • Mechanisms • Training Strategy • Business Impact & Readiness
• IT Operations & Readiness • Training & Awareness
• Stakeholder Management & Communication • Defining Ownership & Accountability
Change
Management
Technology
•
IntegraKng exisKng and Big Data
Technologies, i.e. Master Data
Management
•
Big Data Lifecycle Management
Data Compression & Archiving
Requirements
Regulatory RetenKon Requirements for
Big Data
Business RetenKon Requirements
Data Volumes & Cost
•
Metadata requirements
•
New Sources
4
• Vision & Mission • Objectives & Goals • Alignment with Corporate
Objectives
• Alignment with Business Strategy
• Guiding Principles
• Statistics and Analysis • Tracking of progress • Monitoring of issues • Continuous Improvement • Score-carding
• Policies & Rules • Processes • Controls
• Data Standards & Definitions • Metadata, Taxonomy,
Cataloging, and Classification • Operating Model
• Arbiters & Escalation points • Data Governance
Organization Members • Roles and Responsibilities • Data Ownership &
Accountability
• Collaboration & Information Life Cycle Tools
• Data Mastering & Sharing • Data Architecture & Security • Data Quality & Stewardship
Workflow • Metadata Repository • Communication Plan • Mass Communication • Individual Updates • Mechanisms • Training Strategy • Business Impact & Readiness
• IT Operations & Readiness • Training & Awareness
• Stakeholder Management & Communication • Defining Ownership & Accountability
