Project Level Quantitative Assessment

At Stage 2, the quantitative evaluation should be updated to reflect the specific project and cost environment. The project level assessment covers the period from business case development and approval to advertising the project.

This should be completed in accordance with the Green Book (and SCIM guidance for health projects). It should be collated and reviewed by the Procuring Authority. The variables should be verified by appropriate technical support.

It is desirable that the quantitative VfM assessment of different procurement options is undertaken at a consistent evaluation point, for example typically the forecast projected “contract award” date as the base date. This base date evaluation can continue to be a reference point at Stage 2 and Stage 3 even when the actual contract award date is moved. In assessing VfM at different procurement stages and at different base dates, there is no requirement to undertake and report reconciliations of movements in VfM. Rather, VfM will be assessed between procurement options at a consistent point in time and over an equivalent appraisal period.

As with the Stage 1 discussed above, the quantitative assessment relies upon a sound evidence base, wherever possible built up from past procurement experience in relation to both conventional procurement (e.g. Design & Build contracts, traditional Management Contracts etc) and non conventional (e.g. NPD, JVs).

Where applicable (for projects assessed at the Programme Level), the shadow bid model inputs from Stage 1 will be refined to reflect any project specific changes from the previous review at the Programme Level Stage. Given the high level nature of the Stage 1 assessment, complete new quantitative information may be required for bespoke project appraisal and any assumptions carried forward should be tested. The Procuring Authority should try to understand the drivers behind any significant change in cost estimates between stages 1 and 2.

Sensitivity Analysis

A range of sensitivities should be tested in respect of different input and economic assumptions to assess their impact on VfM. These should identify the key variables which

VfM outcome. In circumstances where conventional funding (e.g. via prudential regime) are part of a potential privately financed solution, the quantitative assessment models used should adopt the same assumptions.

Risk Analysis

Procuring Authorities will need to develop and implement thorough project-specific risk management and risk analysis procedures. In examining projects Audit Scotland and other auditors may wish to examine project appraisals. They might reasonably criticise public bodies that have placed undue reliance on subjective or poorly researched assessments of Optimism Bias and risk as the basis for selecting private finance as the procurement route, or accepting an NPD contract as offering VfM. Authorities will need to ensure that there is effective risk analysis with good evidence to support any adjustments to costings within the VfM assessments.

Conclusions drawn from the quantitative VfM analysis must be based upon detailed sensitivity analysis as point estimates of conventional procurement and non conventional procurement (CPAM / NPD) costs should not be considered in isolation. Likewise, there may be some limitations in some of the qualitative / quantitative assessments that need to be recognised (for example at this Stage, only limited account can be taken of innovation).

Therefore at Stage 2, adjustment to risk quantification and Optimism Bias figures should only reflect evidence based on past experience. In Scotland it is recommended that a bespoke risk workshop is held at this stage to facilitate the pricing of Risk and Optimism Bias. Refer to Section 5 and Annex B. Health projects should also refer to SCIM guidance at: http://www.pfcu.scot.nhs.uk/

Results of the Stage 2 Quantitative Analysis

The quantitative assessments, sensitivity testing and the qualitative analysis will confirm the deliverability of the preferred project scope and procurement route. The quantitative VfM factor is likely to differ by sector.

The qualitative assessment should help the Authority to gauge the level of confidence that can be placed on the quantitative assessment, for example if the qualitative assessment identifies gaps in programme information the Authority may wish to place less reliance on the results of quantitative analysis until this has been addressed.

Procuring Authorities must calculate and confirm their own affordability envelope and compare this with the results of the quantitative analysis. If a project is deemed unaffordable, it should not be pursued.

If Stage 2 analysis shows that privately financed or NPD procurement route no longer represents VfM, the reasons should be reviewed as well as the appropriateness of the procurement route. This may involve re-examination of the project, its scope and allocation of risk. Assessors should consider the following:-

 identify why and whether the issue is specific to the proposed procurement route or to procurement of the project in general;

 consider the case for a delay to the start of the procurement, if this can address the concern;

 review the scope of the project and determine whether there is a better way to deliver the business requirement (e.g. rebuild versus refurbish); and

 reconsider the procurement route and the possibility of switching to other forms of procurement.

When considering the VfM assessment, Procuring Authorities must accord proper weight to the respective parts of VfM analysis - qualitative and quantitative. Providing that the indicative VfM is positive and is based on a balance of combined qualitative and quantitative VfM elements, the project can proceed - there is no necessity to prove quantitative VfM above a particular percentage. The weighting between the qualitative and quantitative elements is a key consideration and should refer to previous evidence bases and qualitative and quantitative assessments.