Performance Measures for Internal Auditing
A simple question someone may ask is “Why measure performance”? An even simpler response would be that “what gets measured gets done”.
McMaster University’s discussion paper on “Performance Indicators” spells out several reasons for performance measurement:
• Clarify the mission and vision of the organization
• Assist in translating its strategy into measurable objectives • Allows the organization to measure progress
• Understand what improves results
• Improved accountability and decision making
• Alignment of operational activates and resources with strategic objectives • Encouragement of dialogue
• A shared understanding of activities planned to deliver objectives • Clear communication of expectations to all organizational levels.
Internal auditing is also susceptible to performance measurement. The Institute of Internal Auditors (IIA) attribute standards 1311 states that:
“Internal assessments should include:
• Ongoing reviews of the performance of internal audit activity; and • Periodic reviews performed through self-assessment or by other persons
within the organization, with knowledge of internal audit practices and the Standards”.
On September 29, 2005, the IIA released Practice Advisory 1311-2: Establishing Measures to Support Review of Internal Audit Activities Performance (Appendix A refers). The Practice Advisory recommends a balanced score card approach be used to track performance measurement into three board categories:
• Stakeholder expectations of Audit Committee, Executive Management, etc • Audit process including risk assessment, planning and audit methodologies • Innovation and capabilities that include the use of technology, training and
industry knowledge.
The performance measure selected by an internal group will depend on the resources available to collect the necessary data in systematic way. Some suggested performance indicators are listed in Appendix B.
Some of the questions to consider include:
• How do we make the performance indicator transparent to our stakeholder? • How much innovation can we afford? To what end?
• Are we in a position to establish professional/ national/regional standards in some categories?
Appendix A
Practice Advisory 1311-2: Establishing Measures to Support
Reviews of Internal Audit Activity Performance
Interpretation of Standard 1311 from the International Standards for the Professional Practice of Internal Auditing
Related Standards:
Attribute Standards:
Standard 1311: Internal Assessments Internal assessments should include:
-Ongoing reviews of the performance of the internal audit activity; and -Periodic reviews performed through self-assessment or by other persons
within the organization, with knowledge of internal audit practices and the
Standards.
Related Practice Advisory:
PA 1311-1 Internal Assessments
Nature of this Practice Advisory: This practice advisory expands on PA
1311-1 to provide guidance in establishing measures for ongoing reviews of the
performance of the internal audit activity. This guidance is not intended to be the only basis for establishing performance measures but is recommended taking into consideration the International Standards for the Professional Practice of Internal Auditing and common measurement practices. While this advisory provides examples of specific measurements considered critical by CAE’s, there is no single set of measurements that is effective for all audit activities.
Introduction
Standard 1310 provides that internal audit activities should adopt a process to monitor and assess the overall effectiveness of its quality program. The process should include both internal and external assessments. PA 1311-1 suggests using the analysis of performance measures as one method for conducting these reviews.
To establish effective performance measurements, the CAE should establish a measurement process that:
• Identifies critical performance categories to measure (e.g. internal customer expectations). This advisory suggests the use of the following categories based on a balanced scorecard approach:
o Stakeholder Expectations
o Internal Audit Processes
o Innovation and Capability
• Identifies performance category strategies and measurements. Strategies are typically based on methods to comply with IIA Standards, other
professional standards, and stakeholder expectations
• Provides an effective ongoing performance measurement and reporting process
• Links to strategies and includes specific baseline and target measurements to monitor progress.
Implementing the use of performance measures, as a method of ongoing review, is one element of the internal audit activity’s internal assessment process. Another key element is periodic self-assessment of the internal audit activity’s compliance with the Standards.
The CAE should ensure that the measures used are appropriate for their activity’s size and their applicable industry, country, national laws and regulations, and operating environment. Performance measures should be specific to the organization and the CAE is cautioned against relying on general measures that are not meaningful to the specific audit activity. Examples of measurements considered important to CAE’s are listed in a diagram at the end of this practice advisory.
Identifying Critical Performance Categories
As noted in the introduction, the CAE should identify key performance
measurement categories such as stakeholder expectations, audit processes, and innovation and capabilities of internal audit.
Stakeholders could include the Audit Committee, Executive Management, external government bodies and regulators, and the external auditors. Audit processes could include risk assessment, planning and audit methodologies. Innovation and capabilities could include effective use of technology, training and industry knowledge.
Next the CAE should determine what objectives to meet in each of the above categories, using the mission, strategy, and organization of the activity. It is important to ensure that the applicable IIA Standards and laws and regulations are identified as guidelines for setting strategies. In addition to the IIA Standards, there may be other applicable professional standards.
Identifying Performance Category Strategies & Measurements
The IIA Standards, other applicable professional standards, the corporate and internal audit activity mission and strategy, laws and regulations, and the internal audit activity charter will provide an effective foundation for determining the appropriate strategies for each category of performance. Performance category strategies and measurements are based on this foundation and an analysis of stakeholder expectations.
Stakeholder Expectations. Typically the key stakeholders for the internal audit
activity are divided into internal and external stakeholders. Internal Stakeholders may include:
• Board/Audit Committee
• Senior and Operating Management External Stakeholders may include:
• Regulators • External Audit
The CAE should identify all relevant stakeholders and perform an assessment of their expectations (and corresponding priorities) and any identified gaps.
Assessments can be performed via interviews, facilitated sessions, and questionnaires. As gaps are identified, the CAE is encouraged to develop an appropriate action plan. The expectations among stakeholders may need to be reconciled and validated.
Considerations in identifying relevant stakeholders and their expectations include:
• The extent of regulation for the organization and/or audit activity • The relationship with the key internal and external stakeholders
• The nature of the organization (publicly vs. privately held, for example).
Internal Audit Processes. The IIA Performance Standards should be
considered in identifying performance measurement categories in internal audit processes. Examples of internal audit process categories and potential areas to measure in each category are:
• Risk Assessment/Audit Planning.
o Are measures in place to ensure that key risk areas are being
addressed?
o Are measures in place to ensure that appropriate managers are
o Percentage of resources allocated to addressing audit committee risk concerns and percentage allocated to addressing senior management risk concerns
o Areas where risk concerns have been mitigated/addressed (reduced
partially, i.e. from severe to moderate) by effective Internal Audit evaluation and comment
o Extent to which audit committee, senior management, and external
auditor believe Internal Audit has effectively addressed risk concerns. • Planning and Performing the Audit Engagement.
o Are measurements in place to ensure that appropriate audit plans are
established for each engagement that include scope, objectives, timing, and resource allocations?
o Are measurements in place to ensure that audits are performed in
accordance with established audit methodologies and working practices?
• Communication and Reporting.
o Is feedback obtained from key stakeholders on the quality, level of
detail and frequency of audit communications?
o Does the internal audit activity measure the degree that key
recommendations are implemented?
Innovation and Capability. The IIA Attribute and Performance Standards
should be considered in identifying performance measurement categories related to the internal audit activities’ innovation and capability. Examples of innovation and capability categories and potential areas to measure in each category are:
• Training.
o Are measurements in place to ensure that audit staff receive sufficient
training (hours of training per staff, completion by staff of critical training subjects, etc)
o Is audit staff satisfaction with training measured?
• Use of Technology.
o Have goals been established for staff training in the use of technology?
Are measures in place to ensure these goals are achieved?
o Have goals been established for using technology to effectively
support audit testing and analysis? Are measures in place to ensure these goals are achieved?
• Industry Knowledge.
o Are measures in place to ensure that the staff has sufficient knowledge
measuring the completion of orientation sessions, audit projects in key areas, working in the operations)?
Below is a pictorial representation of the concepts discussed above in identifying the performance categories and related measures relevant to the internal audit activity. T ra in in g T e c h n o lo g y In d u s tr y K n o w le d g e B o a r d /A u d it C o m m itte e S e n io r M a n a g e m e n t O p e ra tin g M a n a g e m e n t R is k A s s e s s m e n t/ A u d it P la n n in g P la n n in g & P e r fo r m in g th e A u d it E n g a g e m e n t R e p o r tin g R e g u la to r s E x te r n a l A u d it C o m m u n ity C o rp o r a te C u s to m e r P r o fe s s io n a l P r a c tic e s F r a m e w o r k C o r p o r a te a n d In te r n a l A u d it S tr a te g ie s L a w s a n d R e g u la tio n s In t e r n a l C u s t o m e r s E x t e r n a l C u s t o m e r s In te r n a l A u d it P r o c e s s In n o v a t io n a n d C a p a b ilit ie s
Creating an Effective Performance Measurement and Reporting Process
The CAE should establish a measurement process that drives behaviour that supports established audit activity goals/objectives and that provides an
appropriate assessment of achievement of those goals/objectives. An effective ongoing process would include:
• Performance measures that are aligned with the IIA Standards, key strategic objectives, and applicable laws and regulations. These
measures can be both qualitative and quantitative. Measurement points should be clear, measurable, achievable, realistic goals and/or standards • Consistent processes for gathering, summarizing, and analyzing
measurement data and providing timely feedback
• Processes to ensure measures are kept current with changing expectations, conditions, priorities and objectives
• Reporting of the results of the measurement process to department management and key stakeholders
• Annual reporting on the effectiveness of the internal audit activity to the audit committee.
In the example below we have illustrated how the Chief Audit Executive might determine the appropriate performance measurements for innovation and capability: L e v e r a g e c a p a b i l i t y o f k e y s y s t e m s f o r k e y f i n a n c i a l a n d m a n a g e m e n t r e p o r t i n g a s w e l l a s a u t o m a t i n g i n t e r n a l c o n t r o l s . C o r p o r a t e S t r a t e g y L e v e r a g e r e p o r t i n g w i t h i n k e y s y s t e m s f o r a u d i t i n g a p p l i c a t i o n s t h a t s u p p o r t k e y p r o c e s s e s a u d i t e d . I n t e r n a l A u d i t S t r a t e g y M e a s u r e m e n t s : - T r a i n i n g h o u r s p e r a u d i t o r o n r e l a t e d s y s t e m s - N u m b e r o f s t a f f w i t h s y s t e m s a u d i t e x p e r i e n c e P e r f o r m a n c e C a t e g o r y : I n n o v a t i o n a n d C a p a b i l i t y C a t e g o r y S t r a t e g y : R e c r u i t a n d t r a i n s t a f f i n s y s t e m s r e p o r t i n g a n d a u d i t c a p a b i l i t i e s .
Further information for establishing a framework for measuring the effectiveness of the Internal Audit can be found in:
The Quality Assessment Manual published by the Institute of Internal Auditors.
“A Balanced Scorecard Framework for Internal Auditing Departments” by Mark L. Frigo, Ph.D., CPA, CMA. This book was sponsored by the
Institute of Internal Auditors Research Foundation. Attached is an exhibit from this book that includes the “Most Critical Performance Measures when CAE’s were asked to identify the Five Most Vital Measures.”
“20 Questions Directors Should Ask About Internal Audit” by John Fraser and Hugh Lindsay. This paper is available on the IIA website and was sponsored by The IIA Research Foundation, the Canadian Institute of Chartered Accountants, and the Institute of Corporate Directors.
On the following page is an excerpt from “A Balanced Scorecard Framework for Internal Auditing Departments” that lists the measures considered most important by CAE’s.
Staff experience
Training hours per internal auditor
CAE reporting relationship - functional
Percent of certified staff Audit Committee satisfaction survey
Role of internal auditing viewed by audit committee
Audit Committee risk concerns
Importance of audit issue Completed vs. planned audits Number of process
improvements
Number of major audit findings Amount of audit savings Quality assurance techiques developed
Number of repeat findings Days from end of field work to report issuance
Auditee satisfaction survey results Percent of audit recommendations implemented Number of management requests Management expectations of internal auditing
Number of complaints about audit
Professional Practices Framework
Corporate and Internal Audit Strategies
Laws and Regulations Board/Audit Committee
Management and Auditees Internal Audit Process
Appendix B
Some Suggested Performance Measures
STAKEHOLDER EXPECTATIONS
• At least 70% of the time is spent on direct audit hours
• Over a three year cycle, at least 1,000 hours will be spent on:
o Compliance audits
o Consulting
o Information technology audits
o Performance measurement
o Operational auditing
o Risk Management
o Value for money audits
• Fraud Investigations are concluded within 60 days of employee being suspended • Client survey is conducted for every operational audit and results summarized for
the Audit Committee.
INTERNAL AUDIT PROCESSES
• An in-depth risk assessment will be conducted every three years • Risk Assessment profile will be updated annually
• Over a three-year cycle, some aspect of every department will be audited even if no activity is identified in risk assessment ranking
• All audit assignments over 50 hours have an audit plan
• Audit projects are within 10% of the approved budget or maximum of 50 hours over budget
• Operational Audit Reports are issued within six weeks after exit interview • Final reports are issued two weeks after management responses received
• Management implements over 75% of the recommendations within two years of audit report issuance
• Management accepts over 90% of the recommendations in the audit reports. INNOVATION AND CAPABILITIES
• All staff at Associate level and higher will have at least one auditing designation [CIA, CFE, CFI, CISA, etc]
• At least 10% of the staff time is spent on professional development • All staff proficiently use data analysis tools [ACL, IDEA, etc]
• Staff understand the role of Legislation, Regulation, Policy, directives, and program procedures in relation to employees and public.
