Risk management and the achievement of financial VFM

The third issue investigated in Paper 5 is the extent to which Australian auditors- general have investigated risk management and the achievement of VFM in audited PPP projects. Paper 5 indicates that three audits have investigated pre-contracting procedures and processes and the achievement of anticipated risk transfer and financial VFM in the execution stage in two pre-2000 PPPs delivering core correctional and clinical services. (These are the three audits identified as PA7, PA8

and PA9 in Table 3.) A direction from the government (which was disputed by the auditor-general) in the substantive performance audit of the PPP prisons (VAGO, 1999) forced the omission of certain information in the audit report that related to the cost benchmarks established to assess bids submitted by prospective tenderers during the bidding and selection process for each of the private prisons, as well as the actual cost bid submitted by each successful tenderer; payments made to private operators for the periodic delivery of prison services; and amounts deducted to date from payments to operators for poor performance or non-achievement of outcomes specified in contractual conditions (VAGO, 1999, p. viii). However, VAGO 1999 confirms that average per-prisoner daily costs were lower in privately managed prisons than in publicly managed prisons.

118 Auditor-General Oversight Chapter 6

In auditing the implementation of the 1993 NPP, VAGO (1999) confirms that the concept of VFM in the PPP context is far more complex (see also Chapter 5 and the substantive NSW audits into the pre-contracting stage of toll roads) than may have been assumed in the literature (see, for instance, the call for the UK’s NAO to audit non-financial aspects of VFM by Broadbent et al, 2004a). The Victorian audit examined the rights and physical wellbeing of prisoners, the rights of prisoners, prisoner training programs (designed to promote their rehabilitation and successful reparation back into the community), and the quality of training and development programs for custodial staff (see chapter headings VAGO, 1999, p. 33). The list of bodies assisting the auditor-general (VAGO, 1999, p. 35) provides an indication of the level of scrutiny facing the managers of large modern prisons, whether they are publicly or privately managed.

6.5 Chapter summary

Paper 5 indicates that insufficient and perhaps inappropriate oversight of PPPs is provided by Australia’s auditors-general. The overall amount of performance audit has fallen in the post-2000 period. Although it has been argued in Paper 5 that substantive audit of the pre-contracting and the execution stages is necessary, it seems that the primary audit focus has been on the pre-contracting stage, leaving the assessment of achievement of anticipated VFM in the execution stage largely unaddressed. However, this conclusion should be seen in the light of the evaluation of the pre-2000 prisons undertaken by the department and the government.

The classification of performance audits as substantive or systems-based in Paper 5

has indicated that there is a correlation between auditor-general and the type of audit conducted, suggesting that the approach to auditing PPPs could change as new auditors-general are appointed. Also, Paper 5 shows that all of the audits classified as substantive occurred in the pre-2000 period. Additional research is required to understand whether this phenomenon is solely related to particular auditors-general or if, more worryingly, there has been a more subtle change in the approach to performance audit by Australia’s auditors-general in general. Sharma (2007) suggests that VFM auditors (read performance auditors) in England work closely with the Public Accounts Committee [PAC] to bring to the public’s attention to weaknesses in

Chapter 6 Auditor-General Oversight 119

public administration that need to be addressed by the government. Sharma’s (2007) paper suggests that the bland language in VFM reports is deliberate and allows the PAC more scope to more theatrically highlight problems and grill public officials. However, as it is uncertain whether Australian auditors-general and PACs work in the manner described by Sharma (2007). More research into state audit in Australia, its on-the-ground achievements, and the role of various PACs in monitoring PPP performance audits and implementation of audit recommendations, is required.

Clearly, perceptions that recent performance audit of Australian PPPs may be systems-based rather than substantive pose problems for the reputation of auditors- general, and raises questions about their supposed independence from executive government. Heald (2003) has argued that auditors-general are the only source of independent oversight of PPPs with unfettered access to government documents and officials, and with the right to determine how and when to conduct performance audits. In addition, audit reports are publicly available (unlike much of the evaluation undertaken by government departments). Accordingly, the nature and conduct of performance audit is critical to effective parliamentary oversight of executive governments and the effectiveness (or lack of effectiveness) of parliamentary scrutiny of the implementation of major government policies (including PPPs). The evidence presented in Paper 5 suggests that Australian auditors-general have considerable work to do in relation to the oversight of PPPs. Indeed, the concluding section of Paper 5

suggests that recently Parliamentary Accounts Committees have taken steps to fill the oversight role usually undertaken by auditors-general.

Paper 5(p. 332) raises the question of who provides independent assessment of PPPs in Australia. Paper 5 notes that executive government (through their departments of treasury and finance) are the major PPP agenda setters and evaluators, implying that little independent internal examination of PPP outcomes is taking place. Chapter 4 confirmed that, aside from regular facilities management undertaken by departments that is overseen on an exceptions basis by the Department of Treasury and Finance, PPPs have remained largely unassessed at central government level where PPP policies are determined. The Gateway Review Process is designed to fill this gap. Nevertheless, findings presented in Chapter 5 suggest that the Victorian government has commissioned some ad hoc evaluation of PPP projects, and that the Department

120 Auditor-General Oversight Chapter 6

of Human Services (at least) has undertaken significant ongoing evaluation of PPP outcomes through the work of the OCSC and also through reports commissioned to evaluate program implementation (such as the 2002 reducing re-offending reforms). Unfortunately, as noted in Chapter 4, these internal reports are either confidential, or difficult to access.

121

CHAPTER 7

Conclusion

7.1 Introduction

This thesis has three aims, which were outlined in Chapter 1. The first aim was to analyse the characteristics of modernisation in the Australian context. The second aim was to analyse PPPs and their changing role as a modern procurement technology. The third aim was to analyse the oversight of PPPs by Australian auditors-general.

The purpose of this concluding chapter is to summarise the empirical findings arising from the investigation of the three aims of this research, to reflect on the source and nature of differences (if any) between the empirical evidence provided in this thesis and findings in the literature, and to indicate its contribution. Reflecting the iterative reflexive process of theory development in the light of empirical findings, the analysis in this chapter summarises and draws together the major implications of the discussion and conclusions in each chapter.

This concluding chapter is organised as follows. Section 7.2 reviews the research approach. Section 7.3 reviews the motivation of the research. Section 7.4 summarises and reflects on the principal findings in relation to the three research aims. Section 7.5 presents policy implications of the empirical findings. Section 7.6 considers the limitations of this research and outlines future research directions.

7.2 Research approach

As explicated in Chapter 2, this documentary-based study has been undertaken using the middle range thinking research approach developed by Laughlin (1995; 2004). Middle range thinking is congruent with Peraekylae’s (2005) approach to textual analysis. Scott’s (1990) observation that documentary sources are inevitably shaped by their political context and the ideological and cultural assumptions that underlie them is relevant to the analysis in this thesis, and resonates with Laughlin’s (1995; 2004) acknowledgement of the subjectivity of the middle range approach.

122 Conclusion Chapter 7

Five published papers comprised the core of this ‘by publication’ PhD. However, in addition, each chapter reviewed relevant literature to establish the skeletal theory that underpins the investigation of the three aims. According to Laughlin (1995; 2004), there are three stages to the middle range approach. The first process is the a priori

theorising (to develop the skeletal theory), which contextualises the research and provides some guidance as to the empirical points of interest and how to interact with them. The second stage is the empirical stage, which involves gathering the evidence and also reflecting on and modifying (where appropriate) the insights from the skeletal theory in the light of the findings. The third stage of the middle range process is the separate change dimension. As noted in Chapter 2, the change dimension remains under-explicated by Laughlin (1995; 2004). Laughlin suggests that it may involve questioning previous understandings of prior theory, revisions of prior theories, and/or contributing to policy debates.

However, middle range theorists acknowledge that while generalisations about the empirical reality are possible, they are not guaranteed to exist as observed, interpreted and theorised. The subjectivity of the research approach has an additional outcome. It may not be possible for researchers to engage uncontroversially with any of the three stages, giving rise, for instance, to debates as to the nature of modernisation and its achievements in the literature.

In the context of this thesis, the subjectivity (Andon, 2007; Lambert and Lapsley, 2006) of PPP research (including in this thesis) is addressed in Chapter 5, which raises the notion of possible biases in relation to data selection and its analysis. In this thesis, the Victorian government’s decision to introduce PPPs within a particular policy context is taken as a given. Following Laughlin (1995; 2004), choices concerning data selection and the interpretation of data are determined by the ontology of the investigator. The inherent subjectivity of this research approach is acknowledged as a potential limitation of this thesis.

7.3 Motivation

In the light of scepticism about the use of PPPs and their outcomes, the overriding objective of this thesis was to analyse the use of PPPs as a modern procurement