Auditing ERP systems without specific CAATs

(1)

BRAZILIAN COURT OF AUDIT

Auditing ERP systems without

specific CAATs

(2)

Auditing ERP Systems without

specific CAATS

Agenda

 Brazil and IT Audit Secretariat background

 Audit opportunities and risks

 Survey on ERP systems in the Brazilian Federal Public Administration

 Benchmarking of audit methodologies

(3)

Brazil background

 Country data

 5th largest country in the world  6th GDP in the world

 area: 8,500,000 sq. km (2.5 x The European

Community)

 population: 190,000,000 inhabitants

 84th HDI

(4)

 Created in August 2006

 to undertake audits that require specialized

knowledge in IT

 to research, develop and disseminate methods on

IT audit

 to elaborate and provide IT audit training

(5)

Sefti’s Role

 Business: External auditing of information

technology governance in the federal government.

 Mission: To ensure that information technology

adds value to the business of the federal government for the benefit of society.

 Vision: To be a unit that achieves excellence in

improving and auditing information technology

(6)

Auditing ERP Systems without

specific CAATS

 Audit opportunities and risks

 Audit methodology

(7)

 Court Decision

 All of the national energy areas are supported mainly by ERP systems

 Company #1 (SOX Compliance)

revenues in 2010: US$ 118,3 bi

 Company #2 (SOX Compliance)

revenues in 2010: US$ 15,2 bi

(8)

 Lack of knowledge of auditors regarding the topic

 No prior audits on the topic carried out by TCU

 Lack of a support tool (CAATs) to audit controls related to the application of ERP systems

(9)

Auditing ERP Systems without

specific CAATS

 Survey on ERP systems in the Brazilian

Federal Public Administration

(10)

Survey

 57 national public companies

 Most in the energy business (Petroleum and Electricity)

 49% of them use ERP systems and 33% plan on using ERP systems in the medium term

49% 18%

Respondents by category

Use Plan

(11)

 3 main suppliers

 SAP is the leader, followed by Totvs (a national

company) and by Oracle

Survey

36% 25%

Supplier Quantitative Distribution

SAP Totvs Oracle

(12)

 Cost of acquisition of licenses and customization approximately US$ 666 million

 Scope of benefits from implementation of ERP system

Survey

0% 20% 40% 60% 80% 100% Information Security Work process Management issues Benefits Categories

(13)

Auditing ERP Systems without

specific CAATS

 Benchmarking of audit methodologies

(14)

Benchmarking

(Experientia Mutua Omnibus Prodest)

 INTOSAI Readings

 IntoIT Issue 27, December 2008

 Assuring SAP (Australia)

 IntoIT Issue 28, April 2009

 Dutch Experiences with ERP Systems

 Country Focus South Africa

 19th Meeting of Intosai Working Group for IT Audit (WGITA)

 SAP in public administration (Netherlands)

 Visits

 RMAS (Risk Management & Audit Services) at Harvard University

(15)

Auditing ERP Systems without

specific CAATS

 Audit methodology

(16)

Audit methodology

 Five companies selected

 Company #1 - (SOX Compliance) revenues in

2010: US$ 44,4 bi

 Company #2 – (SOX Compliance) revenues in

2010: US$ 15,2 bi

 Company #3 - revenues in 2010: US$ 7 bi

(17)

 Audit Scope

 Focus on evaluation of general controls, due to the

lack of a support tool for evaluating application controls

 Use of globally accepted audit criteria (Cobit 4.1,

ISO 27.002, ISO 31.000, ISO 15.999) and national legislation

 10 audit questions associated to 49 possible

findings

 Survey with 9,000 users from the selected

Audit methodology

(18)

Dimensions Audit questions

MANAGEMENT OF ERP SYSTEM AND IT PLANNING

Q1. Is management of the ERP system based on IT plans and policies?

Q2. Is a cost-benefit analysis of the investments in the ERP system carried out?

PROCESSES AND METHODS OF SUPPORT

Q3. Do the professionals who support and use the ERP system undergo appropriate training and receive

information that is appropriate to carry out their activities? Q4. Does the IT area count on processes and methods to support the ERP system?

PERFORMANCE OF THE INTERNAL AUDIT

Q5. Are the management and use of the ERP system overseen by internal audit?

CONTRACTS AND LEGAL

ASPECTS Q6. Do the contracts related to the ERP system meet the _{legal provisions?} INFORMATION SECURITY

CONTROLS

Q7. Have the general IT controls associated with the

security of the ERP system been implemented according to best practices?

(19)

Findings Q9: User satisfaction

Less than 1 year 3% Between 1 and 3 years 12% Between 3 and 5 years More than 5 years Did not respond 0%

(20)

Findings Q9: User satisfaction

24%

42%

5%

Distribution of length of time using system

Use the ERP system more than other systems

Use other systems more than ERP system

Use ERP and other

(21)

Findings Q9: User satisfaction

73%

14%

9%

4% 0%

Influence of system use

Increases my productivity

Does not influence my productivity

Decreases my produtivity I don´t know

(22)

Findings Q9: User satisfaction

38%

61%

1%

Need to reenter ERP system information in other systems

Yes

No

Did not respond

1%

Need to reenter other systems information in ERP system

(23)

Findings Q9: User satisfaction

12%

47%

33%

8% 0%

General level of satisfaction with system use

Totally satisfied

Very satisfied

Partially satisfied Dissatisfied

Did not respond

The system is not trustworthy 2% The system is frequently offline 3% _{The system}

does not have the operations I

need 11% The system is Did not respond

22%

(24)

Auditing ERP Systems without

specific CAATS

(25)

 It is possible to audit ERP systems without the use of specific CAATs

 The steps suggested are:

 Carrying out a survey on the status of ERP use in

the country

 Benchmarking of audit methodologies

 Carrying out survey among users of the systems of

chosen companies

(26)

 If the SAI does not have previous experience or resources to acquire specific CAATs to help in ERP system audit, it should invest in

knowledge and motivation in order to face the challenges of a task of such importance

(27)

Thank You!

[email protected] 55 (61) 3316-5371