Normally, the evaluation of proposals is undertaken by an appointed evaluation committee. The formation of an evaluation committee is described in Section 15.2. It is important that the evaluation committee is appointed early and that all necessary arrangements for carrying out the evaluation work in a confidential manner are in place before the date for the opening of proposals.
Evaluations in which the financial offers are considered in the comparisons of proposals may be considered in three stages: the first stage checks the eligibility of each applicant and the completeness of the proposals; the second stage considers the engineering and managerial capabilities of the applicants; the final stage compares capabilities with the financial offers.
1st stage: Eligibility, completeness of proposal and financial stability
While the eligibility check would have been carried out for prequalification, it is advisable to check again to ensure that circumstances have not changed.
An initial task is also to check the completeness of each proposal to ensure that all documentation required has been correctly submitted.
Thereafter, it is necessary to check the financial
stability of each applicant. Again, this check would have been made previously for prequalification, but it is inevitably worthwhile to ensure that no adverse movement has occurred to the financial status of an applicant.
The committee should be satisfied that the applicant has the financial resources and backing to meet salaries, wages, materials, supplies, and all other costs until the forecast payments become due under a consultancy services agreement and to meet all other ongoing commitments during the period of the consultancy services.
As a guide the committee will expect the total annual turnover of the applicant to be on the order of at least three times the anticipated annual turnover of the services. Failure to meet the minimum financial requirements should result in the applicant being excluded from further evaluation. These checks complete the first stage of the evaluation.
The results of the first stage analysis of eligibility, completeness and financial stability can be conveniently recorded on an evaluation matrix form by marking each applicant as either ‘s’ for suitable or ‘u’ for unsuitable against the described criteria, an example of which is shown in Table 16.1. Any applicant marked as unsuitable against any criterion should be rejected at this stage.
164
16.3 Evaluation of proposals with price comparison
Information to be provided to the consultant prior to the start of negotiations should include:
- The scope of services required.
- Terms of reference and supporting documents.
- List of additional information to be provided by the consultant.
- Date by which an agreement is to be concluded.
- Expected start date and programme for the provision of the services.
- Arrangements, as may be required, for visiting the site of the proposed works.
All the criteria considered at the time of prequalification (see Chapter 7) should be reviewed in depth during the negotiations to ensure that written prequalification information submitted by the preferred consultant is not only comprehensive but up-to-date.
The negotiations for the fee structure will be based on an agreed scope of services which takes into consideration commercial and legal requirements, including time schedules, payment terms and appropriate risk allocations between the parties.
The client may refer to fee schedules by various FIDIC Member Associations and/or client bodies to assist in the negotiations.
If agreement cannot be reached with the highest ranked prequalified applicant, the negotiations shall be terminated and commenced with the second ranked applicant, and so on down the list until agreement is finally reached.
16 Evaluation and Recommendations:
Consultancy Appointments
Notes 1 An ‘s” is marked for suitable and a ‘u’ for unsuitable.
2 Any tenderer with a ‘u’ mark against any criterion should be rejected.
3 In this example, all tenderers are taken forward to Stage II.
Table 16.1 – Evaluation of proposals for consultancy appointments
1st stage -Eligibility, completeness of proposal and financial stability Matrix example: suitable/unsuitable marking
Proposal Evaluation Criterion
Eligibility and Competence of the Tenderer Financial Stability Available resources: design capability Services to be subcontracted
Experience: relevant projects completed Experience: relevant projects in progress Experience: geographical
General suitability: language
Quality assurance system; health and safety; environmental policy
Notes 1 Scores are the average scores of all evaluators for each proposal under each criterion.
2 The effect of the application of the weighting percentages is shown in Table 16.3.
3 The criteria and the weighting percentages to be applied are examples only: the particular circumstances of the proposed project will define the requirements for possible alternative or additional criteria and weightings to be included in a proposal dossier.
Table 16.2 – Evaluation of proposals for consultancy appointments 2nd stage - Engineering and management capability Matrix example: average of evaluators’ scores
Total:
2nd stage: Engineering & managerial capabilities
Careful comparison of the proposals is then required in the second stage of the analysis to evaluate the engineering and managerial capabilities by use of a subjective scoring system to rank the applicants in order of merit.
A convenient method of comparing proposals in the second stage analysis is to record subjective scores against criteria representing various aspects of the submissions. The resultant scores are then suitably weighted. An example of use of this method is given in the evaluation matrices shown in Tables 16.2 and 16.3.
The criteria in the example are:
- Organisation and management - Available resources: design capability - Services to be subcontracted
- Experience: relevant projects completed - Experience: relevant projects in progress - Experience: geographical
- General suitability: language
- Quality assurance system; health & safety; environmental policy
In Table 16.2 and 16.3 it is considered for this example that the subcontracting of services would be
166
16 Evaluation and Recommendations Consultancy Appointments
detrimental to the qulaity of services to be provided and, accordingly, has been given a negative weighting. While negative weighting and scoring will not generally be necessary, the example demonstrates how negative weighting may sometimes be appropriate.
Where appropriate and feasible, discussions may be held with applicants. This provides an opportunity of meeting and assessing the personal capabilities of key personnel who would be intimately associated with the project.
The criteria above are shown on the left-hand side of the matrices while columns are provided for entries of the scores against the criterion under each of the applicants numbered 1 to 5 (five applicants are considered in this example). The scoring against each criterion is in the range 0 - 100, 100 being ‘exceptional’
and going down through the ranges of ‘very good’,
‘good’, ‘average’, ‘poor’ and ‘weak’ to 0 showing ‘no data’.
It is good practice for the same evaluators, who may be more but never less than two, to score each criterion for each proposal and for the evaluators’ scores for each separate criterion for each proposal to be averaged. This method avoids possible distortion in scoring values, which might be caused by different evaluators marking relatively high or low. To assist in the comparison of proposals, it is also recommended that the evaluators score all proposals for a particular criterion before the next criterion is considered.
Each criterion will have been previously weighted. The average score against each criterion for each proposal is then adjusted by the weighting
percentage. In the example given in Table 16.3, the effect of the weighting percentages is shown in the matrix. The criteria and weightings may, of course, be adjusted to suit the circumstances of the project.
3rd stage: Capabilities and proposal prices
For scoring comparison with the proposal sums, the engineering and management scores are then calculated as follows:
-the proposal with the highest percentage is given a technical score of 100;
-the other proposals are scored by the ratio:
Technical score of proposal under consideration
= [(weighted score of proposal under consideration)/
(weighted score of best technical proposal)] x 100
From the engineering and managerial scores shown in Table 16.3 the best three proposals, namely 3, 4 and 5, are short-listed and taken forward for
comparison of their financial scores.
Consideration has now to be given to the financial aspects of the proposals. It is necessary to check that the fee structures, on which the rates and prices and totals of the proposal sums have been submitted, are acceptable for comparison purposes.
Applicants should be advised of any arithmetic errors in their proposals. Any other clarifications should be sought from the applicants as necessary, such as, for example, the reasons for any abnormally low-priced proposals.
From the analyses of the proposal sums the assessed effective total proposal prices from the applicants are assessed. The assessed total prices are then calculated as follows:
-the lowest assessed total price is given a financial score of 100;
- the other proposals are scored by the inverse ratio:
Financial score of proposal under consideration
= [(lowest total assessed total price)/
(assessed total price being considered)] x 100
Using the method of scoring explained above, the financial scores and their relative rankings are shown in Table 16.4. The most economically advantageous proposal is identified by combining the evaluated engineering and managerial scores with the financial scores on a 90/10 basis.
In the example shown in Table 16.5 it will be seen that, after adding the engineering and technical scores multiplied by 0.90 to the scores of the financial scores multiplied by 0.10, applicants numbered 3 has the most economically advantageous proposal.
However, it is also to be noted that this applicant is neither the highest ranked applicant in the engineering and managerial evaluation nor the applicant with the lowest assessed total price.
16 Evaluation and Recommendations:
Consultancy Appointments
Table 16.4 – Evaluation for consultancy appointments 3rd stage - Financial scoring
Applicant Table 16.5 – Evaluation for consultancy appointments
3rd stage - Matrix example: combined scoring
Applicant Code
Notes 1 The proposal with the highest average technical and managerial weighted score, applicant 4 in this example, is given a final evaluation score of 100 and the remaining applicants are given a final engineering and managerial
score in the ratio of (actual average weighted score/highest average weighted score) x 100, as shown below.
2 In this example the three highest scoring applicants numbers 4, 3 and 5, are short-listed and taken forward for comparison with the financial scores.
Proposal Evaluation Criterion
Organisation and management Available resources: design capability Services to be subcontracted
Experience: relevant projects completed Experience: relevant projects in progress Experience: geographical
General suitability: language
Quality assurance system; health and safety;
environmental policy
Total average weighted score:
1
Table 16.3 – Evaluation of proposals for consultancy appointments 2nd stage - Engineering and management capability Matrix example: application of evaluators’ scores
1
Managerial Score x 0.90
73.01
In this example, all applicants are taken forward for
comparison and evaluation with their engineering and
management capability.
Note
In this example, applicant 3 is considered to be the most economically advantageous proposal.
16 Evaluation and Recommendations Consultancy Appointments
168
Upon commencement of the consultancy services, the client should promptly notify the other successful prequalification applicants held in reserve (for
a quality-based selection process) or other applicants in writing that their proposals have been unsuccessful.
In the unlikely event of rejection of all applications and proposals, the applicants should be advised of the reason. If new proposals are to be invited, the client should carefully review the causes leading to the rejection and should consider modifying the prequalification documents or proposal dossiers or adding to the list of applicants before inviting new proposals. It is also recommended that any new proposal dossiers should be updated to incorporate any modifications previously issued as addenda to proposal dossiers and/or replies to applicants’ queries.
Proposals should not be rejected and completely new proposals invited against the original proposal dossiers for the sole purpose of trying to obtain lower prices.
Subject to any particular conditions or regulations governing the formation of agreements, if the client is
considering the rejection of all proposals because the lowest priced proposal exceeds the cost estimates or the client’s budget by a significant amount, the client may negotiate with the most favourably placed applicants.
In conducting such negotiations, the guiding principle must be to ensure that the confidentiality and fairness of competition is preserved. In the event of a satisfactory agreement not being negotiated, all proposals may be rejected.
Rejection of all proposals is also justified when:
- proposals are not substantially responsive to the issued proposal dossier; or
- too few proposals are received to ensure competition.
16.5 Rejection of all proposals
16.6 Notification to unsuccessful applicants
Upon completion of the quality-based selection process or, alternatively, where price is an element in the selection process, a report of recommendation for the award of a consultancy services agreement will normally be prepared for submission to the client by the
evaluation committee.
The report will contain:
- a summary of the recommendations followed by a detailed text of the object, method and conclusions of the quality-based selection negotiations;
or, where a number of proposals have been analysed:
- proposals opening (as appropriate) and initial review;
- evaluation process/procedure;
- reasons for rejection of proposals; and - recommended proposal with reasons for
recommendation.
If the client accepts the recommendation he may proceed to prepare and sign an agreement for the required services with the recommended consultant.
In the process of awarding the consultancy agreement, relevant provisions set out in Chapter 19 concerning the award of contracts for works may be applicable. However, it is not normal for the consultant to be asked to provide a performance security as described in Section 19.5.