Aligning with more of the ‘classic’ waterfall delivery model, the following section sets out some multidimensional ‘checklist’ questions that might be utilized in a review of project at a stage gate or checkpoint.33
The stages that we describe are generic and so may not represent any specific organization:
1. Concept and feasibility; 2. Requirements and architecture; 3. Build;
4. Testing, acceptance and implementation; 5. Post-implementation.
We first discuss the rationale for each of these stages and then look at the man- agement team’s set of reasonable expectations. Specific items that contribute to the risk schedule and help identify issues at each stage are included in Appendix 1.
Concept and feasibility
Good ideas can fail to get effectively addressed and many do not proceed beyond the initial ideas phase because project outputs are not sufficient to justify a management decision to release funds and allocate resources to the project.
Identifying, reporting and managing project risks 99
33 While we recognize the dangers of relying on checklists (i.e. leave your brain at the
door) the research of Heemstra and Kusters (1996) found that in the context of IT projects the use of a short and structured checklist eases the identification and discussions about risks. They also suggest explicit use of a group-related approach and involvement of a neutral process risk advisor to increase reliability and acceptance of the results. Of course, items not on the checklists should also be considered.
Alternatively, we sometimes see that funds are committed and a project commences without addressing the critical questions.
Adequate concept and feasibility effort will result in adequate definition, justification and buy-in, and a clear basis on which to make the ‘go’ / ‘no go’ decision upon completion of this phase.
At this stage we are expecting to find:
•
A clear business rationale and clarity of concept;•
A focused and effective assessment of solution and delivery options to confirm feasibility;•
Clarity of commercial value and demonstrated achievement of mandated investment criteria;•
Appropriate consideration of wider organizational impact, stakeholder engagement and business process changes; and•
A mobilized and effective project team working to accepted standards.Requirements and architecture
Many business projects can become IT projects at this stage. The focus on specifications, analysis, technologies and the need to increasingly communicate in IT rather than business terms, can result in a loss of business support.
Detailed requirements definition often ‘blows out’ the original expectations of project scope. The link back to business priorities and objectives can be lost and a suboptimal solution specified.
Vendor selection can be an extremely drawn-out and costly process – often exacerbated for government organizations by procurement and probity requirements. Lack of rigour in project start-up may take its toll here as poor control over project costs and timeframes becomes more visible to management. Unplanned risks and issues emerge to thwart progress.
At this stage we are expecting to find:
•
Continuing relevance of the project to the business community that will benefit from its completion;•
An optimized technical solution and vendor delivery commitment in line with acceptance criteria agreed with the business;•
Commercial arrangements that tie solution delivery partners to meaningful milestones and business achievements rather than technical deliverables;•
A more closely calibrated business case underpinned by detailed estimatesand quotes for the technical solution and vendor delivery approach;
•
A basis for ongoing and tight integration with the necessary business and organ- izational changes that must accompany the introduction of the IT solution; and•
A combined business, IT and vendor management and delivery team that is performing well to agreed standards and staying on top of emerging issues and risks.Build
Whether a buy or build project, this is the project phase where the service provider responsibilities dominate. Project activities can become ‘bogged down’ in technical issues and details.
Further drift from the original business objectives can occur. Suppliers focus on delivery to defined statements of work and must continually be drawn back to focus on overall project intent.
Interim milestones are frequently missed or ‘passed on a technicality’. Suspected rework can build up – waiting to be revealed during testing. Confidence in the ability of the supplier to deliver to the original budget and timeframe weakens.
Often the parallel business activities to equip the organization in readiness for change become misaligned with the IT work stream.
At this stage we are expecting to find:
•
Continuing business support and involvement in the project and continuing relevance of the initiative to the business it will enable;•
Confirmation of the solution as fit-for-the-purpose in all dimensions spanning the full scope of functional and non-functional requirements;•
An agreed and suitably detailed test plan that, when executed, will underpin achievement of the agreed solution acceptance criteria;•
Service provider and vendor delivery against commitments;•
Tight management of project costs; and•
Continuing high performance of project teams.Testing, acceptance and implementation
Nasty surprises are typically ‘discovered’ late. This is often the case during accept- ance testing where the full solution is assembled and real-business scenarios are executed for the first time.
True defects are difficult to sort from the mountain of test incidents that are generated. As the rework cycle takes its toll an adversarial mindset can develop: the solution provider is ‘guilty until proven innocent’ and tasked with the enormous burden of fixing functionality only fully specified when the detailed test case was written.
Ambiguity around acceptance criteria and disputes over late and undeclared changes to the agreed functionality surface and prove difficult to resolve – for whether borne by supplier or customer there is significant cost involved.
At this stage we are expecting to find:
•
Active involvement of business representatives to bring rigour and relevance to the testing activity and assist in readying for implementation;•
Demonstration of the solution as fit-for-the-purpose in all relevant functional and non-functional dimensions;•
Clear management of service provider and vendor outcomes through to release from obligation in line with achievement of meaningful milestones and acceptance criteria;•
Application of commercial rigour to drive delivery in line with forecast costs – without pushing costs and pain into the post-implementation period;•
Organization readiness for change associated with system implementation; and•
Effective teamwork across organization boundaries towards a shared and clearly articulated set of goals.Post-implementation
With a sigh of relief the project is typically declared over when the systems go live.
Despite the attention to detail in the formulation of a benefits realization plan, the reality is that the project team is often disbanded and insufficient resources and attention allocated to driving out the planned benefits.
An independent project review can help focus the project team on achieving:
•
Business ownership and attention to follow-through on the implementation ofbusiness changes that will realize benefit;
•
Active measurement and monitoring of solution performance to ensure agreed service levels are met and service provider and vendor commitments under warranty are met;•
Satisfactory progress towards a ‘business-as-usual’ state in the operations, support and maintenance activities;•
Closure of project accounting and a ‘ruling off of the books’; and•
Identification of lessons learned and incorporation into a revision of guidelines, principles, methods and approaches.Appendix 1 provides terms of reference for health checks that would be commissioned by a steering committee or governance body towards the goals above. Additional issues, risks and stakeholder concerns that are specific to the initiative would also shape the scope, approach and detail of each review.
Health check
This section assists you consider at a macro-level the company’s IT project risk health, for input into the IT risk portfolio assessment.