Document Analysis: Published and Unpublished Documents

4.4.1.1VFM Audit Reports

The study reviewed a sample of 20 of 136 VFM audit reports that were submitted to the Victorian Parliament by the VAGO from 1982 to 2007 (see Appendix 4 and 5). The sample of 20 VFM audit reports was selected by adopting the following criteria.

1. All 3 VFM audit reports submitted by Waldron, Auditor-General (1976-1985) to the Victorian Parliament to reflect the details of emergence and policies of the VFM audit from 1982 to 1985.

2. A random sample of another 17 VFM audit reports, stratified to reflect the different time periods and different Auditors-General in order to review and monitor the development of VFM audit methodologies, corporate planning and audit environment from 1985 to 2007.

As Gendron et al. (2007) advises, these VFM audit reports were selected because they serve as mirrors to the VFM audit. The VFM audit reports provide evidence of the gradual development of VFM audit scope, objectives, technology and reporting style over the 25- year period examined in the study. Further, the VFM audit reports sampled were selected to reflect the work of five different Auditors-General who held positions during the research period, thereby addressing the possibility raised by Hamburger (1989) that the personality and practices of the Auditor-General explain VFM audit practices.

In addition to the sample of 20 VFM audit reports, the researcher reviewed six recent VFM audit reports with the relevant computer audit files and hardcopy files to track the development of the VFM audit methodology, known as the AmP, implemented by the VAGO in 2007. The details of the AmP computer audit files reviewed are presented in Table 4.2 below.

Table 4.2: Review of VFM Audit Methodologies and Files

Date Name of the audit file Days spent Audit

methodology

1. 25–27 February 2008

Southern Cross Station Project 3 days AmP 2. 17–19 November

2008

Victoria’s Planning Framework for Land Use and Development

3 days AmP

3. 14–16 April 2009 Patient Bed Flow Management 3 days AmP 4. 12–14 June 2009 Patient Flow and Bed Management in

Public Hospitals

3 days AmP

5. 23–25 October 2010

Tendering and Contracting in Local Government

3 days AmP

6. 6–8 June 2011 Managing Water Infrastructure Projects in Australia

3 days AmP

The review of the VFM audit reports and AmP methodology was relevant to answer the research questions of how audit methodology was developed, as reflected in VFM audit practice, and how those audit methodologies changed their role from being an accounting device to being a powerful inscription and possibly having agency role.

The VFM audit reports were a rich source of information in elucidating the research themes, such as the relationship between NPM and the VFM audit, development of the VFM audit mandate, Auditor-General’s independence, idiosyncratic influence of the

Auditor-General, development of VFM audit methodologies, creation of the VFM audit environment suitable to VFM audit knowledge, and development of the VFM audit scope. For each of the audit reports included in the sample, the researcher scrutinised the cover letter, executive summary, detailed audit recommendations and audit agency responses to understand the audit findings and recommendations. The cover letter to the Victorian Parliament provided the VFM audit mandate, whereas the executive summary provided the foreword by the Auditor-General, scope of the audit, audit findings and recommendations, and auditee responses to the audit findings. Thereafter, the researcher prepared a data reduction summary report for each VFM audit report and highlighted research themes and data codes. The research themes and data codes are presented in Figure 4.1.

4.4.1.2Annual Plans and Annual Reports

The researcher reviewed seven annual plans and twenty three annual reports published by the VAGO from 1985 to 2007 in order to answer the research questions regarding how VFM audit methodologies were enacted and operationalised within the audit environment, as reflected by strategic planning in an Auditor-General’s Office. The annual reports provided information such as audit policy and methodological developments, summaries of corporate plans, brief summaries of VFM audit reports and financial audit reports submitted to Parliament. They also provided information about what the VAGO had achieved during that particular year. The annual report outcomes could be compared with the annual plans of the VAGO. The first annual report was submitted in 1984 as per the Annual Reporting Act of 1983. In accordance with the Audit (Amendment Act) 1999, which required the VAGO to submit its annual plan to the PAEC and obtain its comments, the VAGO commenced submitting the annual plan to the PAEC from 2000. The researcher reviewed the seven annual plans submitted to the PAEC from 2000 to 2007 to identify sections relevant to the research themes. Subsequently, the relevant sections were summarised under the research themes and a data summary report was compiled for each annual report and annual plan.

4.4.1.3Corporate Plans

This study reviewed seven corporate plans completed by the VAGO from 1986 to 2007 in order to examine the major developments and operationalisation of audit methodologies and other in-house practices. The corporate plan works as a script and inscription of the impending drama that will be performed by the VAGO for a three-year period.19 This study expects that the corporate plans drive and manoeuvre the audit environment, ensuring its suitability for the audit methodologies already developed by the Auditor-General’s Office. The corporate plans can predict the behaviour of the VFM audit actors, which are the Parliament, PAEC and auditee organisations. This study also expects that the VAGO also aligns their interests with those of the VFM audit actors and stakeholders through corporate plans.

This study expects that the corporate plans to change the behaviour of other actors, such as the PAEC and Parliament, because corporate plans work as communicating devices (scripts and inscriptions) moving from the VAGO to the actors and audit environment, such as the PAEC and Parliament. The VAGO completed its first five-year corporate plan in 1986 and has since completed five corporate plans up to 2007. The corporate plans were reviewed to identify important paragraphs relevant to the production of audit reports, the development of VFM audit methodologies and other backstage practices and procedures. Subsequently, these paragraphs were summarised into a data summary report.

4.4.1.4Parliamentary Acts

The audit acts enacted by the Victorian Parliament since 1857 were reviewed. These audit acts provided the audit mandate and other provisions in relation to auditor independence and the reporting responsibilities of the VAGO. The audit acts function as the binding relationship between the actors and audience (such as Parliament and PAEC) in the VFM audit drama. The VAGO commenced VFM audits through the mandates given by the Audit

Act 1958, Audit (Amendment) Act 1990, Audit Act 1994, Audit (Amendment) Act 1997 and

Audit (Amendment) Act 1999. The PAEC conducted a review of the Audit Act 1994 and

released its recommendations to the Parliament in 2010.

4.4.1.5Unpublished Reports and Other Documents

This study evaluated the documents maintained by the library of the VAGO in order to plan the data collection sources. Based on this evaluation, the researcher reviewed unpublished reports, such as the Public Bodies Review Committee Report published in 1981 (VAGO 1981). The other unpublished records included internal staff reports completed by staff members of VAGO who had undergone VFM audit training programs in Canada (VAGO 1984b). The PAEC’s reports and performance audit reports compiled by the independent reviewer appointed by the PAEC to evaluate the effectiveness of the VAGO for every three years were also evaluated, because these reports could provide vital information about the performance of the VFM audit comparable to other Auditor-Generals’ Offices and professional accounting firms.