Presented by: Fred Wechselberger
Global Accounts Manager, CaseWare Analytics
Control Testing & the Relevance of Data
Today’s Agenda
• Audit and Present Day Analytics:
– Planting seeds to provoke thinking over time
• The Visionary Audit Department:
– Encouraging you to think a little differently about something now
• Data Analysis Insights: Case Studies
Provoke thinking over time
“Trust but verify”
“Modern Scientists are doing too much trusting and not enough verifying”
“In the biotechnology venture-capital about half of the publish research can not be replicated.”
“Knowing what is false is as important as knowing what is true”
Ever Changing Risk Environment
The world is constantly changing – new risks are always emerging:
• Economic pressures and challenges
• Extreme weather
• Population growth and aging
• Political instability
Internal and external risks are continuously evolving:
• Businesses are more dynamic and agile
• Regulatory changes and increased in enforcement
• Talent that can drive and
navigate competitive advantage • Fraud, waste and abuse
Hard to keep up with the speed of growth, change, and use of technology:
• Cyber threats • Privacy issues • Social media engagement and reputation impact • Big Data
Source: IIA Pulse of the Profession - Defining Our Role in a Changing Landscape Report
Are you reactionary?
• Businesses are less strategic with their spend
• Traditionally, audits have been conducted on a
retrospective basis
• Internal Audit acted as an enforcer of business
controls
Largest Credit Union Fraud
• On September 27, 2011, Anthony Raguz, the former chief operating officer of the St.
Paul Croatian Federal Credit Union (FCU), pleaded guilty to six counts, including
bank fraud, money laundering, and bank bribery, for his role in one of the largest credit union failures in American history.
• Raguz issued more than 1,000 fraudulent loans totaling more than $70 million to
over 300 account holders in the Albanian and Croatian communities near Cleveland
from 2000 to 2010.
• He accepted more than $1 million worth of bribes, kickbacks, and gifts in exchange for the fraudulent loans. Raguz is one of 16 people charged for their roles in the credit union’s collapse.
• The failure of St. Paul Croatian FCU resulted in a $170 million loss to the National Credit Union Share Insurance Fund
.
FBI Financial Crimes Report to the Public Fiscal Years 2010-2011
Proper SOD and Data review could have
revealed what was going on
What happened
– Loans issued without requiring collateral – Borrowers known to have no assets
– No employment history – Use of fictitious names
– Use of fictitious companies – Bribes paid
Fraud starts small
• Barings bank was destroyed by $1,724 million in bad debt . The fellow started by covering a 40k loss for a friend.
• LA School District number of years ago lost millions on purchases made just under a limit. ( 5 years )
• Duplicate payments made to vendors on the same invoices first in Canadian dollars then in US dollars. ( 3 years)
Are you reactionary?
3 times as many frauds caught by fraud hot lines.
Firms with hot lines suffer less from Fraud.
More Corporate and Government support for whistleblowers.
SEC $450M fund to reward whistleblowers
Periodic Audits
C ont rols Ef fec tiv enes s Time Audit 1 Audit 2 Audit 3Source: Continuous Auditing From a Practical Perspective, Kevin Handscombe
13
Actual Expected Effectiveness
Continuous Audits
C ont rols Ef fec tiv enes s Time CA CA CA CA CA CA CA CA CA CA CA CASource: Continuous Auditing From a Practical Perspective, Kevin Handscombe
14
Actual Expected Effectiveness
Source: IIA Pulse of the Profession - Defining Our Role in a Changing Landscape Report
Stakeholder Perceptions
Value-Added Insights
Are you a visionary?
• Businesses are more dynamic and agile
• Insights have moved from what happened to what is happening
• Risk-based auditing aligns with overall organizational risks • Internal Audit is a value added resource
Primarily driven by data analytics
All you need is data
“In God we trust; all others bring data”
Source: p.31 Barry Beracha former CEO Sara Lee Bakery Group
“…Oracle, IBM, Microsoft and SAP…” 15 B
Creating Key Insights
• Universal view of critical business risks
• Detecting controls breakdown and deficiencies
• Changes in critical risks and controls before the
business is impacted
• What is being done by the business to remediate
the issues
Relevant and Timely Insights
• Focused on areas of highest risk
• Detecting changes to the environment
• Ability to isolate key processes, locations,
people, etc.
• Timeliness enabled by technology
• Primarily driven by data analytics
• Quantifiable impact
How do you get to be the Visionary?
• Quantifiable impact
• Good knowledge of business process
Pick something you know well
Risk Objective Ordering Req/PO Splits Req/PO Duplicates PO before Requisition Limits Exceeded PO Timing PO Approvals Pricing Changes SoD FRAMEWORKGovernance, Risk & Controls Definition and Monitoring
Remediation – Routing, Alerts, Follow-up, Escalations, Comparison
Reporting and Visualization – Exceptions, Compliance, Dashboards,
Metrics P R O C E S S F L O W Receiving Received vs. PO PO Timing Overdue Goods Delivery Timing SoD Invoicing Deviations Invoice vs. Received Duplicate Invoices Price differences Retroactive Invoices Suspicious Invoice #’s SoD Payments Split Payments Payment vs. Invoice Duplicate Payments Prohibited Vendors Top 100 Lists Excessive Payments Manual Payments SoD Inventory Turnover Analysis Dead Stock Vendors Master Data Related Parties
Very objective
Low Risk Objective Risk
• Risk Matrix with Controls Tests Frequency
High risk very subjective
Judgmental high risk Risk
• Quantifiable impact
• Good knowledge of business process • Data available and understood
Example
Corporate Department a Section a Section b Department b Section c Monitoring team 175 control groupsPhase one – report exceptions to IA
Phase two expand out to the operational groups
New
1. Sources:
• Data dumps • Report files • ODBC
1. Sources: • Data dumps • Report files • ODBC 2. Tools: • ERPs • CAATs • ETL Tools
Data Access
1. Sources: • Data dumps • Report files • ODBC 2. Tools: • ERPs • CAATs • ETL Tools 3. Types: • Transaction vs. Master Data • New data or Pulling Everything
Data Access
Data Access
Data Quality
Nestle ten year project :
The first step is improve accuracy 100,000 products
200 countries 550,000 suppliers
9 million records re vendors, customers, materials.
1/2 where obsolete, duplicates
1/3 of the reminder where inaccurate or incomplete.
Data Quality
Nestle is not alone:
Most CIOs admit that their data are of poor quality ( IBM Study)… half the managers don’t trust the data
•Firm to be more efficient…
•America operation saved 30 M a year on vanilla
•Total saving of 1.B/yr.
Data Quality
Definition:
Data has quality if it satisfies the requirements of its intended use.
Jack E.Olson
“Data Quality” The Accuracy Dimension
Morgan Kaufmann Publishers
Data Quality
Correctness Time sensitive Meaning full Complete Confidence Two banksBottom up
Errors in the application, reporting, or understanding
Correct data transforms into incorrect data as it is used for different purposes
or decays over time.
Errors compound as the data is consolidated
Data element
Birth Date
Correct form, content and context
Correct form is mm/dd/yyyy Correct date is January 3, 2006 Correct context Jill’s birthday
Wrong form
03/01/2006 dd/mm/yyyy ( Cdn or Europe)
Wrong content
10/03/2006 Transposed the month 01/30/2006 Transposed the day 09/11/2006 Entered the wrong date.
Wrong person (context)
Analytical
Techniques Form
Account No A765
Character field length 4
Valid upper case Alpha A – N , P-Z Valid Numeric 0-9
Mini value A001 max value Z999
Content
Is the data correct
Element analysis (a field in isolation):
Analytical Techniques
Form
Invalid data
Content
Structural analysis Value correlation
Aggregation correlation Value inspection
Analytical Techniques
Rules that define
relationships between columns
Rules that define
relationships between tables
Analytical Techniques
Rules that define relationships
between the fields
The values must be true over the range of data
Analytical Techniques
Are the values reasonable • No clear boundaries • No Limits • No rules work Visual inspection • Frequency • Comparing to other data sources • Natural thresholds • Text Strings
Value Inspection
Analytical Techniques
Does the data pass the test of
“Common Sense”
Examine aggregated values of large data sets
• Quantifiable impact
• Good knowledge of business process • Data available and understood
• Used CAAT to perform audit
• Quantifiable impact
• Good knowledge of business process • Data available and understood
• Used CAAT to perform audit • Tests can be automated
• Some tools are better than others but use what you have to get going
• Dump exceptions into a central repository
• Scripts should use source data and exceptions repository to determine recurrence and eliminate duplicates
• Use parameters/variables to determine how the logic works to prevent changing the script each time
• Some of the simplest scripts yield the greatest business value
• Building libraries of tests • I tunes for auditors
• Private libraries • Public libraries
• Maximum window (A)
• Timeline between control breakdown and impact (B) • Time to resolve the exception (C)
• A = B + C
Deadlines and Auto Transitions
Explicit Deny Access
The rule of law
Rank Country/ Territory BPI 2008 Score Respondents Standard Deviation Confidence Interval 95% Lower Bound Upper Bound 1 Belgium 8.8 252 2.00 8.5 9.0 1 Canada 8.8 264 1.80 8.5 9.0 3 Netherlands 8.7 255 1.98 8.4 8.9 3 Switzerland 8.7 256 1.98 8.4 8.9 5 Germany 8.6 513 2.14 8.4 8.8 5 Japan 8.6 316 2.11 8.3 8.8 5 United Kingdom 8.6 506 2.10 8.4 8.7 8 Australia 8.5 240 2.23 8.2 8.7 9 France 8.1 462 2.48 7.9 8.3 9 Singapore 8.1 243 2.60 7.8 8.4 9 United States 8.1 718 2.43 7.9 8.3 12 Spain 7.9 355 2.49 7.6 8.1 13 Hong Kong 7.6 288 2.67 7.3 7.9 14 South Africa 7.5 177 2.78 7.1 8.0 14 South Korea 7.5 231 2.79 7.1 7.8 14 Taiwan 7.5 287 2.76 7.1 7.8 17 Brazil 7.4 225 2.78 7.0 7.7 17 Italy 7.4 421 2.89 7.1 7.7 19 India 6.8 257 3.31 6.4 7.3 20 Mexico 6.6 123 2.97 6.1 7.2 21 China 6.5 634 3.35 6.2 6.8 22 Russia 5.9 114 3.66 5.2 6.6Confidence
Pastor gets a year in jail for defrauding church
Premier of Alberta resigns over expenses
Out reach program for drug addicts in question
Senate scandal over expenses
Fraud Triangle
Opportunity
Need
Example
• The whole entity is corrupt
– Day 1 on the job.
– Human Skull on fence post with photo copies death certificate stuffed in the mouth.
– Watched some one receive cash in return for documents.
Case for monitoring
New Corporate entity
Amnesty “ yesterday does not matter”
Clear rules and policy
New pay system - reward on merit “Help Money”
Promotion on merit
Dismissal for cause
Moving Forward
• Be creative about how you approach data and
analysis
• Invest in the people and tools necessary for
success
• We live in a ocean of data you have to learn to
swim.
• Start from the risks
Fred Wechselberger [email protected] 1.800.265.4332, ext. 2807 Maxwell McCone [email protected] 416.366.7227
