Introduction
Quality control procedures applicable to the audit process itself are a vital element in seeking to assure high quality in the work of the SAI. Effective quality control procedures – based on international standards and the experience of other countries – are described in the first part of this chapter. Establishing such procedures is the first step. In addition, an SAI must assure that the procedures are operating effectively. The best means of doing so is the use of post-audit quality reviews. This is addressed next in this chapter. However, other SAIs have learned from long experience that effective quality control procedures – vital though they are – are only part of an overall strategy for building a fully effective SAI, producing high quality audits and having an appropriate influence on the accountability and effectiveness of state entities. This part of the overall strategy can be described as “managing the institution for quality”. Good practices in this broader area are discussed in the latter part of this chapter.
Quality Management in the Audit Process
The nature and extent of an audit organisation’s internal quality control system depends upon factors such as type of the Supreme Audit Institution (Court of Audit/Accounts or National Audit Office), size, degree of delegation of decision power, nature of work, type of audit carried out, organisational structure, appropriate cost/benefit considerations, level of staff, legislation, etc.
Levels of Quality Management
There are three levels of quality management. On the first level, the SAI should adopt and implement policies and procedures designed to ensure that audit tasks in all phases of an audit are carried out to an acceptably high level. These quality control procedures centre on the management of the audit process, that include the elements of direction, supervision and review. This would aim to confirm, on a ‘real time’ basis (called hot reviews, i.e. during the actual conduct of an audit), that integral quality monitoring processes of ongoing audits have operated satisfactorily, quality of audit reports is ensured, and repetition of weaknesses is avoided.
On the second level, the SAI should carry out higher level quality assurance reviews by SAI staff or independent experts who have not been directly involved in the audits. Although this level of quality control is after the fact (called cold reviews), it is closely associated with the first level of control as it aims to ensure that ‘real-time’ controls are actually being applied and are effective. This second level of control also helps ensure that ‘real-time’ controls are not relaxed by management or by the auditors directly involved in the audits. Quality controls are therefore continuously kept in force and improved upon.
On the third level, the SAI should from time to time provide for an external review of their operations by senior auditors from other SAIs (or equivalent), a so called Peer Review (cf. the SIGMA Peer Reviews). This is a big undertaking for both parties and a recommended time interval might be five years. This
element of quality management is not discussed in detail in this report. It is, however, highly recommended that SAIs arrange for and carry out such reviews.
(Sources: No 51 “Quality Assurance” of the European Implementing Guidelines for the INTOSAI Auditing Standards; No. 2.1.27 and 2.1.28 of the INTOSAI Auditing Standards; Module B. 12 “Quality Control and Quality Assurance” of the ECA Manual; IFAC International Standard on Auditing on “Quality Control for Audit Work”)
Phases of Audit - Planning, Execution, Reporting
Planning, Execution and Reporting are the three phases of an audit. An audit quality control system should be in force during all three stages. Some SAIs separately include a further phase, called the “Follow-up”, in which a follow-up of the audit findings and recommendations is conducted in a stipulated manner and within a specified time-frame. Other SAIs include the follow-up in the Planning stage of subsequent audits. For the purposes of this report, follow-up is not treated as a separate phase of the audit.
All persons involved in an audit have a personal responsibility to understand their duties and to ensure that their work is carried out up to standard. This involves all phases of an audit. It is also important that quality control takes place in an atmosphere of openness and trust - an interactive process where subordinates also are allowed to give feedback to superiors on the way the audit is managed and present ideas of possible improvements.
All levels of staff should be assessed on a periodic basis to determine any skills, qualifications and aptitude gaps of the staff. During this assessment process one may also find out whether things have gone wrong (and why) during the audit and what corrective action may be taken to rectify these shortcomings in future audits.
Managing the Audit Process - Direction, Supervision, Review
Heads of SAIs and their managers should ensure that they have appropriately high levels of management skills and knowledge to ensure, ‘inter alia’ that effective Direction, Supervision and Review are present during all three stages of an audit.
Direction
This involves the appropriate direction of staff to whom work is delegated. Auditors are informed of their responsibilities and the objectives of the procedures they are to perform. They are informed of the nature of the entity’s business and possible accounting or auditing problems that may affect the nature, timing and extent of an audit procedure in which they are involved. Direction to audit staff is provided through meetings, informal oral communications, audit manuals and checklists, audit programmes and annual audit plan. Audit staff should be encouraged to raise any questions they may have with more experienced team members.
Supervision
This involves monitoring the progress of the audit to consider whether auditors have the necessary skills and competence to carry out assigned tasks; whether they understand the audit directions; and whether work is being carried out in accordance with the annual plan and audit programme. Other supervisory
duties include becoming informed of, and addressing, significant accounting, data collection and auditing questions raised during the audit, assessing their significance and modifying the annual audit plan and audit programme as appropriate. Also supervised are other elements of the audit process, depending upon the type of audit and resolving differences of professional judgement between personnel.
Review
This is performed by more senior staff of appropriate experience to consider whether work has been performed in accordance with the audit programme; work performed and results obtained have been adequately documented; any significant audit matters have been considered; appropriate consultations have taken place; the objectives of the audit procedures have been achieved; and that conclusions expressed are consistent with the results of the audit and support the audit opinion.
Also reviewed on an appropriate basis will be the audit plans and audit programmes; assessments of inherent, detection and control risks; the documentation obtained from substantive procedures and conclusions drawn thereon, the draft reporting and other elements of the audit process, depending upon the type of audit.
It is also important that audit work files carry evidence of all reviews (involving at least two levels in the hierarchy) that have taken place.
(Source: Annex 6 “Purpose and Scope of Audit Management” of the Prague Recommendations; UK Auditing Practices Board – revised SAS 240 “Quality Control for Audit Work”)
Top management should ensure that Direction, Supervision and Review are done in such manner as to provide reasonable assurance that work has been performed competently.
Responsibility for quality control starts with the individual auditor in the Audit Team and then passes up through the hierarchy: the Audit Team Leader, Audit Director and the Senior SAI Management5. The Head of SAI performs the final review and approval of the Audit Plan and Audit Reports, prior to their submission to Parliament.
In an SAI that is organised in a collegiate system (such as Courts of Audit), decisions involving the different processes in audit work, particularly the reporting stage, are often carried out on a collegiate basis.
Modern IT Audit Systems
Modern audit systems that make extensive use of computers to render the audits largely paperless, including the related working papers and documentation, contain many in-built controls and safeguards. In such systems all stages of the audit could be in an electronic format.
An automated quality control system may incorporate a strictly defined set of authorization and approval criteria, as well as features that ensure that standard documents and checklists (which may be electronically readily available to all audit team members) are used and completed in all cases. With such systems some
5 The organizational structure and titles of positions vary widely among SAIs. For convenience in this chapter,
Team Leader is intended to denote the senior member of the audit team and the person who directs the day-to- day work of the team, Audit Director denotes the supervisory level immediately above the Team Leader, and Senior SAI management the level immediately above the Audit Director.
parts of the work of supervisors and reviewers are electronically supported on a real time basis. A first draft of an audit report can be generated automatically, following the conclusion of the audit. One must point out here that ultimately, a system, including an IT system, is only as good as what is put into it.
Quality Control
Sources for Quality Control Criteria
Quality criteria should ensure that all the three phases of the audit are carried out in terms of the SAI’s rules, practices and procedures. Quality control measures should also ensure that audits are timely, comprehensive, adequately documented, performed and reviewed by sufficient and suitably qualified staff and based on sound professional judgement. It should also be ensured that audit work is based on fact and evidence. Intuition, hearsay and opinion must be supported by solid evidence.
Reference to, and criteria for, quality control could be found in a number of sources, as: • Audit Legislation;
• Auditing Standards (including INTOSAI Auditing Standards - European Implementing Guidelines; IFAC International Standards on Auditing; and SAI’s own Standards);
• Mission Statement of the SAI and other SAI Policies; • SAI Regulations, Circulars, Checklists and Guidelines; • Audit Manuals.
Quality in Planning
While the names may differ from one SAI to another, the planning process6 should include several stages, starting from Strategic Planning, Corporate Planning, Operational Planning, Budgeting, detailed Audit plans and the development of detailed Audit Programmes. Within the context of this report, planning refers to the detailed audit plans and subsequent audit programmes. No reference is being made to the macro level of planning that an SAI may carry out.
The preparation of an audit plan should take into account risk and materiality, based on an understanding of the auditee and its business. The plan should set out how and when the audit will be conducted and how sufficient appropriate audit evidence is obtained in order to enable conclusions to be drawn and support the audit opinion.
6 Terminology differs from one SAI to another. The output from this process can be termed Plans or
Typical Contents of an audit task plan 7 1. Legal Framework for the audit
2. Brief description of the activity, programme or body to be audited (including a summary of the results of previous audits and their impacts)
3. Reasons for the audit
4. Factors affecting the audit, including those determining the materiality of matters to be considered
5. Risk Assessment 6. Audit Objectives
7. Audit Scope and Approach: what evidence is to be obtained to meet the audit objectives; when; how?
• Materiality thresholds;
• Systems to be evaluated and tested • Sampling strategies
• Anticipated sample sizes;
• Reliance on other auditors/experts; • Any special problems foreseen;
• (Methodologies referred to further on in the Execution Phase of an audit should be adequately planned).
8. Resources required, and when: • Audit staff (in detail), responsibilities; • Specialist staff (who and when); • External experts;
• Travel requirements; • Time and cost budgets.
9. If appropriate, an estimate of the fee to be charged for the audit 10. Details of those within the audited entity responsible for the liaison 11. Timetable for the audit, and the date that draft report will be available
for internal consideration
12. Form, content and users of final output.
7 “European Implementing Guidelines for the INTOSAI Auditing Standards” – No. 11 Audit Planning, Annex
Participants
The Audit Plan should ideally be based upon consultation among SAI personnel who are involved in the specific audit. This may involve the Head of SAI and/or his Deputy who would approve the annual Audit Plan, the responsible Senior SAI Management member, the Audit Director and, at least, the team leader or the more senior audit team members who are involved in the specific audit. This sharing of responsibility enhances the accountability of all the main players involved in the audit. Furthermore, the sharing of experiences at different levels of the organisation would help ensure a more accurate and complete Audit Plan and more realistic Audit Programmes.
Requisites and Measures to ensure Quality Control
The Audit Plan is divided into a number of detailed tasks, which are assigned to individual team members. To ensure quality control during the planning process, measures could include direction, supervision and review procedures to ensure that the audit task plan referred to earlier is adequately carried out.
Primary measures may include an SAI management policy for the use of standard layout and structure of documents, following the requirements of International Auditing Standards (INTOSAI and IFAC), and the completion and approval of a planning checklist.
The responsible Senior SAI Management member should ensure that the Audit Director has suitably taken into account:
• the financial and human resources required (such as audit staff responsibilities, specialist staff and experts required, training, travel arrangements, time and cost budgets) for the audit. Allocation of available resources are based on the priorities of the SAI;
• the timetable for the audit, and the date the draft report will be ready for internal consideration;
• follow-up issues of previous audits. Audit Permanent Files should clearly indicate which previous audit issues and recommendations need to be followed up. This is usually done during the Planning phase of a subsequent audit, although Follow-up may also be dealt with separately from the three phases of Planning, Execution and Reporting/Follow-up;
• other factors that effect the project management of the audit.
Before the beginning of an audit, a pre-audit meeting could usefully be held between the Audit Director, team leader and team members in order to discuss issues that are to be audited, ways to carry out the audit, as well as to discuss potential problems that may be encountered.
The Audit Director should identify the team leader and ensure that he/she and audit team members have:
• been clearly assigned roles and responsibilities;
• appropriately considered follow-up of observations and recommendations made in previous audits; • updated permanent audit information, including audit issues arising previously;
• obtained comprehensive understanding of the legal framework for the audit, and taken into account any legislative requirements and changes;
• obtained sufficient understanding of the activity, programme or body to be audited;
• adequately considered factors affecting the audit, including risk and materiality assessments; • adequately considered the audit objectives;
• properly understood and undertook the audit scope and approach (including materiality thresholds, systems to be evaluated, sampling, reliance on experts);
• undergone appropriate ongoing liaison with the auditee as necessary with respect to the timing, scope and approach of the audit;
• ensured that the planning assumptions remain appropriate, taking into account any significant events occurring after approval of the plan.
There should be written approval of any changes in the audit planning tasks by the Audit Director. Changes that substantially effect the overall audit plan, as to objectives of audit, timing, human and financial resources required should be endorsed by the responsible Senior SAI Management member.
Audit Planning should also include monitoring of job by the Audit Director as to budgeted against actual time spent for each job, based on job sheets filled in by each auditor. Monitoring costs is important to assess whether an audit is developing well or not.
An independent review of audit proposals and programmes by peers or other Units within the SAI may also be a useful method of undertaking a Quality 'Hot' review of the Planning process.
Annex 1 of the Questionnaire (see Appendix 1) provides a number of guidelines on what audit supervisors and reviewers could look into to ensure quality control in audit planning. Work of reviewers should be documented.
Quality in Execution
Typical Methodologies for Carrying out Audit Tasks
The fieldwork has to be performed in accordance with the approved audit plan and should result in sufficient appropriate evidence being obtained to determine with reasonable confidence whether or not the financial statements are free from material misstatement and irregularity or that facts relating to VFM audits are scientifically and/or fairly arrived at.
The following methodologies and practices are used in the execution of audits. All references are to the European Implementing Guidelines.)
1. Sources: Methods and Nature of the Reliability and Evaluation of Audit Evidence include (Annex 1 of No 13)
Sources:
Generated by auditor directly Obtained by third party
Methods:
Inspection Observation
Inquiry and Confirmation Computation
Analysis of financial systems Nature
Documentary, visual or oral (the reliability of oral evidence, in particular, depends upon the source)
2. Audit Approach includes (Annex 2 of No 13) Objectives
Regularity (financial and compliance)
Value for Money (economic, effective, efficient) Testing
Systems Based Approach (testing of internal controls) Direct Substantive Testing
3. Study and Examination of Internal Control and Tests of Control (No 21) 4. Information Systems (No 22)
General Installation Controls (Annex 1 of No 22)
Planning,Staffing, Reporting and Segregation of Duties Security awareness and policy of both hardware and software Continuity and Disaster Recover
Management of IT Assets and use of External Service Providers Application Audits (Annex 2 of No 22)
Organisation and Documentation Input
Processing
Data Transmission
Standing Data
Output 5. Audit Sampling (No 23) 6. Analytical Procedures (No 24)
Trend Analysis
Ratio Analysis
Predictive Analysis
7. Using the Work of other Auditors and Experts (No 25) 8. Documentation (No 26)
This is particularly important for supervision, review and quality assurance. Working Papers – current and permanent files; Confidentiality; Retention Procedures
9.Performance Audit Methodology (Annex 1 of No 41) Data Gathering Techniques:
File Examination
Audit Sampling
Secondary Analysis/Literature Search Surveys
Interviews
Focus Groups
Benchmarking
Before-After Studies
(Quasi-) Field Experiments/Experimental Method Techniques for Information Analysis
Programme Logic Model
Descriptive Statistics to understand Data Distributions
Regression Analysis
Cost-Benefit Analysis
Cost-Effectiveness
Meta Evaluation
The choice and means of conducting the above methodologies should also be actively considered and studied during the Planning Phase of the audit.
Participants
The main participants in this phase of the audit are the responsible Senior SAI Management member, the