Review of Shared Services Canada’s
Collaborative Procurement
Process
Presentation to the Information Technology Infrastructure
Roundtable
May 2015
Yves Genest, Chief Audit and Evaluation Executive
Office of Audit and Evaluation
Agenda
Overview of the Review
Overview of the Collaborative Procurement Process
(CPP)
Summary of Findings:
Interviews with Information Technology Infrastructure
Roundtable (ITIR) members
CPP file review
CPP vendor feedback
Literature review
Conclusions
Appendix
2Overview of the Review
Objective of the review:
To identify opportunities for improvement
Examination phase: December 2014 to March 2015
Methodologies employed:
Interviews with ITIR members File review of CPP projects Analysis of vendor feedback Literature review
[Additional details on these methodologies, as well as the review limitations and mitigation strategies, are provided in the Appendix.]
Not an audit or evaluation
Overview of the Collaborative
Procurement Process
Developed at Shared Services Canada (SSC), building on Public Works and Government Services Canada’s (PWGSC) National Shipbuilding
Procurement Strategy and used for SSC’s complex procurement projects.
Refined by initial consultations with the information and communications technologies (ICT) sector (2011 to 2012).
Aligned with SSC’s vision to support value for money by closer collaboration and ongoing discussion with the ICT sector.
Characterized by:
engagement with industry early in the procurement process (pre-Requests for Proposal, or RFP) to co-develop requirements;
generally shorter timeframes for the procurement process (from launch to contract award).
Overview of the Collaborative
Procurement Process (Continued)
The chronology of the typical CPP:
Summary: Interviews with Information
Technology Infrastructure Roundtable members
Key Themes:
Engagement with Vendors
SSC Capacity
Innovation
Small- and Medium-sized Enterprises
Suggested Best Practices
Engagement with Vendors:
Overall, the CPP is viewed positively and as an improvement in engagement with vendors, compared to the traditional
procurement approach.
One-on-one meetings are viewed as the CPP’s most helpful vendor engagement mechanism.
Industry Days and Requests for Information are perceived more as platforms for SSC announcements.
ITIR members indicated that there is no feedback on vendors’ input provided throughout CPP phases.
A few ITIR members noted that the vendor community could better align with SSC’s needs by having greater access to relevant SSC planning documents.
7
Summary: Interviews with Information Technology
Infrastructure Roundtable members (Continued)
Summary: Interviews with Information Technology
Infrastructure Roundtable members (Continued)
SSC Capacity:
ITIR members recommended that SSC increase its participation at innovation and trade events.
Some members recommended keeping and developing leading-edge expertise in-house to drive procurements.
ITIR members would like to see SSC explore opportunities to leverage the existing procurement capacity of other
governments / departments and vendors (possibly through partnerships with them).
SSC’s procurement model is viewed as conservative, rigid and risk averse (as is the case with government procurement
generally).
Summary: Interviews with Information Technology
Infrastructure Roundtable members (Continued)
Innovation:
Innovation procurement was recognized as a challenge for SSC and the CPP.
There are programs that test innovative technologies within government (PWGSC’s Build in Canada Innovation Program).
However, none are available to procure innovative services and products.
As a way to support innovation, SSC could incorporate requirements for innovation in RFPs.
Other opportunities could involve an innovation testing program within SSC, which could:
serve as a ‘test bed’ for potential CPPs prior to their launch; and focus on end-state business requirements.
Summary: Interviews with Information Technology
Infrastructure Roundtable members (Continued)
Small- and Medium-sized Enterprises (SME):
SMEs make up an important part of the ICT sector in Canada.
SMEs in Canada have demonstrated the capacity to innovate in various areas related to IT.
SSC would benefit:
by fostering more integration of SMEs in its procurements; and through a wider distribution of risk across project components.
This could be achieved by:
parcelling large procurements into integrated and layered components; and including clauses encouraging vendors to identify how SMEs will be
leveraged.
Summary: Interviews with Information Technology
Infrastructure Roundtable members (Continued)
Suggested Best Practices:
Increasing clarity about the end-state requirements for large IT procurements and possibly breaking down procurement into manageable pieces.
Focusing on the overall value and merit as opposed to lowest cost.
Favouring low bids does not always result in lower costs, as low bids may be a reflection of the amount of risk a vendor is willing to take.
Vendors may recoup funds through other means, such as “managing change orders”.
Incorporating proven innovative procurement solutions in other jurisdictions (provincial, U.S. government), including:
set-asides for innovation pilots and SMEs;
the provision of ‘seed’ money to vendors for bid preparation; and the inclusion of a proof-of-solution stage prior to contract award.
Measuring and reporting on procurement results.
Summary: File Review
The Office of Audit and Evaluation (OAE) obtained files from SSC Procurement.
Selected six CPP projects that had advanced the furthest through CPP phases.
Reviewed files against Gartner’s best practices. Results validated with procurement officers.
12
CPP Project Status
Email Transformation Initiative Awarded / Implementation
Data Centre Consolidation Services Awarded / Implementation
GCNet Wide Area Network (WAN) Services Awarded
High-performance Computing Renewal Services RFP
Data Centre Server Storage Infrastructure RFP
Summary: File Review (Continued)
The CPP has evolved and adapted across projects.
There is variability in how CPPs are designed and managed, which reflects the flexibility and adaptability of the procurement process in relation to vendor feedback and project requirements over time.
There is also variability in bid assessment/evaluation approaches (e.g. 80/20 versus 10/90 split between financial and technical
criteria across CPP projects).
Evaluation criteria and their weighting methodology in CPP projects are very complex.
Summary: Vendor Feedback on File
Overall, vendors felt that CPP projects followed procurement steps. Some vendors noted that one-on-one meetings were effective. SSC
can make improvements in notifying vendors of changes made to documents, as well as in making announcements to vendors in a timely manner.
Most vendors expressed concerns about the heavy weighting on the financial components of the evaluation criteria, versus the technical components. They also had concerns about the financial evaluation methodology itself.
Some vendors noted that the process was long and costly.
However, they also commented on the insufficient time allocated, at certain stages in the process, to provide feedback to SSC and
prepare proposals.
Summary: Literature Review
Early engagement with vendors provides significant opportunities for collaboration and supports value-for-money procurements.
Complex and unique solution procurements benefit from weighting
technical components over financial components and an initial test to proof solution and cost.
A fixed-price or price-cap approach tends to favour value for money and provides vendors with opportunities for innovation.
Parcelling of large procurements into integrated components allows for more effective risk management. In this approach, the use of a contract manager has demonstrated positive outcomes.
Open and flexible procurement solutions are required to support innovation and SMEs.
Clear governance, ownership and accountability of procurement processes and reporting on value achieved ensure a fair, open and transparent
procurement.
Conclusions:
Do you agree?
Overall, the CPP is viewed as a step in the right direction to promote effective collaboration with industry. The process is seen as open and inclusive.
Enhancing reciprocity and communication in the relationship was
identified by ITIR members and vendors as an area for improvement. The CPP’s potential was perceived as not being fully realized in past
CPP projects because the collaborative aspect in defining RFP
requirements with the prospective vendors was, ultimately, secondary to the role played by the financial assessments of the bids.
Among the suggestions for strengthening procurement capacity were: participation in industry and innovation forums, leveraging
external capacity, supporting innovation, including a proof-of-solution stage in the procurement process, and focusing on best value.
Appendix: Review Methodology
Methodologies employed by the OAE:
Interviews with past and current, internal and external ITIR members (n=11);
File review of SSC projects that have used CPP* (n=6);
Analysis of vendor feedback provided in post-award debriefs and post-RRR surveys that were available on file (n=7);
Literature review to identify best practices in procurement. The OAE did not conduct interviews with the CPP projects’
technical and procurement authorities. Meetings with
procurement staff were held to validate file-review observations.
*Projects that had advanced the furthest through CPP phases were selected.
Review Limitations and Mitigation Strategies
18
Limitation Mitigation Strategy
CPP is a new and continually evolving process. The small number of CPP projects and the
uniqueness of each CPP made comparison and generalization across projects difficult.
Results were analyzed for high-level themes, which were
corroborated across all
methodologies for consistency. Each ITIR member represents a different group
of the ICT sector and has a different experience with CPP, bringing a unique perspective.
Some ITIR members shared not only personal opinions, but also feedback received from
vendors or industry associations.
Comments were grouped into broad themes (e.g. vendor engagement, innovation) and were reported as summaries.
Secondary references (i.e. vendor opinion) were identified.
Some vendors chose not to provide feedback to SSC.
Some vendor feedback available on file was consolidated by SSC Procurement rather than documented verbatim.
All vendor feedback available on file was incorporated into analysis. The OAE may conduct a vendor survey to update the findings of this review in the future.
