Our Data Analytics Journey, Methodology, and More. September 15, 2015

Our Data Analytics Journey,

Methodology, and More

Objectives

High-level Objectives:

• Discuss Audit Data Analytics History

– Industry

– Personal History – TIAA-CREF History

• Define our data analytics integration process

• Discuss how to “consume” data analytics and mitigate consumption risk

– Read / interpret results and follow-up procedures and questions – Discuss results with the business

– Best practices and common pitfalls

Industry History

• Late 1980s – generalized auditing software companies form

– ACL, 1987

– Caseware, 1988

• Charles Carslaw, Applying Benford’s Law to Accounting, 1988

• Continuous Process Auditing System, AT&T Bell Laboratories,

1989

• Continuous Monitoring Platform

– Audit Exchange 2.0, 2004

Personal Journey

Late 90’s / Early 2000’s “Cutting Edge” Technology

• DB2

• JCL

• Easytrieve

• Oracle v 8.0

• SQL Server v 6.5

• Microsoft Access

• ACL v 6.5

• ACL for MVS

• Cold Fusion

Personal Journey

Data Analytics Mission and Team

To be a progressive, collaborative and proactive data analytics function that supports risk identification and monitoring processes, integrated audits,

continuous auditing, Division reporting, and proactive fraud reviews and investigations.

Data Analytics Team:

• Tim Penrose, Managing Director, Joined IAD October 2010. • Brian Allen, Director. Joined IAD in July 2013.

• Brian Karp, Manager. Joined IAD in January 2014.

• Lindsay Holden, Senior Data Analyst. Joined IAD in July 2015.

Current DA Tools

Diverse and Evolving Toolset:

• Internal Audit Data Mart

– Microsoft SQL Server 2012

• Visualization Software

– Tableau Desktop Professional 9.0 and Tableau Server

• Internal Audit Data Analytics BI Portal

– SharePoint 2010

• Statistical Software (e.g. R and SAS) • Big Data Tools

– Teradata Aster – Splunk

• Desktop Generalized Auditing Software

How do we do this?

DA Integration Process - Planning

• Scope & Objective Definition Stage

– Identify and document the scope and risks associated with the engagement and communicate that plan to the audit client.

• Business Requirements Definition Stage

– Attend walkthroughs, engage audit partners, and develop a DA test plan that aligns to the process, risks, and controls in Team Mate.

• Data Acquisition Stage

– Request and obtain primary and secondary data sets independently, from IT and/or the business.

DA Integration Process - Consumption

• Understand what the results tell you and the related risks.

• Understand the logic that got us there and why we might have false positives.

Follow-up procedures:

• What do we do next?

• Discuss internally and refine results • Include result items in sample testing

• Follow-up directly with the business to discuss what we are seeing in the data

Tips for discussing results with the business:

• Engage early

• Proceed with caution

Consumption Risk

Reputational Risk with Business Management and within Audit Division:

• Results are not properly understood or vetted prior to approaching management, which erodes trust

• Incorrect conclusions drawn from data • Incorrect results in report

Audit Risk:

• Potential exceptions are not identified by DA or are identified by DA and not analyzed/evaluated by Audit Team

Definitions

• Data analytics is defined as the process of inspecting, cleaning, transforming, and modeling data with the goal of highlighting useful information, suggesting conclusions, and supporting decision making.

-Various sources

• Data analytics is an analytical process by which insights are extracted from operational, financial, and other forms of electronic data internal or external to the organization. These insights can be historical, real-time, or predictive and can also be risk-focused (e.g., controls

effectiveness, fraud, waste, abuse, policy/regulatory noncompliance) or performance –focused (e.g., increased sales, decreased costs, improved profitability) and frequently provide the “how?” and “why?” answers to the initial “what?” questions frequently found in the

information initially extracted from the data.

-KPMG

Audit Data Analytics

Four areas of Audit Data Analytics:

• Audit and/or Investigation Support

− Help Desk Incidents example − Link Analysis example

• Self Service

− Investigation Self Service Dashboard

• Internal Audit Process (Professional Practices)

− PPG Dashboard

Help Desk Incidents

• Frequent emails sent internally notifying users of Sev 1 and Sev 2 system outages

Questions Asked:

• What is the cause of these issues?

• Are these issues occurring more frequently than usual?

• Who is affected by these issues (internal or external customers)? • Are these incidents related to a particular line of business?

Tool Selection:

Link Analysis

• Device and IP address information was collected from 18 involved participants with confirmed online fraud activity. The data was filtered for known fraudulent indicators.

Question Asked:

• Are these IP addresses and Device IDs connected?

• If so, what is the relationship between these IP addresses and Device IDs?

Tool Selection:

• Teradata Aster

Self Service

• During the course of an investigation, our Investigators may need specific pieces of customer information promptly.

Question Asked:

• Can you provide information about a specific customer? • Do multiple customers share the same information?

Tool Selection:

PPG Dashboard

• Current process of creating and maintaining monthly PPG dashboard is time consuming and cumbersome

Question Asked:

• Can you automate and improve the PPG dashboard process? • What additional metrics can be created to monitor audit statuses?

Tool Selection:

tiaa-cref.org

Brian J. Karp, CIA, CISA, CFE, CRISC

[email protected]