Lessons Learned Log CiCS Programme & Project Unit






Full text


Lessons Learned Log

CiCS Programme & Project Unit

This is a summary of key points which have been picked up in Lessons Learned Reviews which were undertaken at the end of many CiCS-run projects over the years.

• The source of each item is identified – CiCS staff can get the background by looking at the Lessons Learned Review documents on the shared drive in Projects/Project Reports.

• The items have been edited where appropriate to fit with this format, whilst retaining the meaning.

• They have been broken up into areas of interest, but inevitably there is much overlap.

This log is not of course a comprehensive guide to good project management, but hopefully it will spread good practice and help projects avoid pitfalls which previous projects have encountered. Many points reinforce each other, whilst others may be quite contradictory, reflecting the balancing act which is often required of projects.



• Although challenging, the [initial] review phase of a project is very necessary. [Environmental Printing Review 2009]

• Brainstorming at the start was helpful. [Environmental Printing Review 2009]

• At the start of a project have open discussions allowing radical thinking. Challenge preconceptions, including overall approach, benefits and timescales. Start with what we want to achieve. A system should an enabler, not a driver. [Supportworks for Service Management 2012]

• Allow lots of time for working on completely new areas – it’s slow to start. [Java 2003]

• If a project starts 'mid-stream' the lack of proper start-up can mean that key decisions are taken on the hoof. A review of decisions made or assumed, key issues, scope, user needs etc is needed at project start-up, even if the work is already under way. [MS Office 2007 Upgrade 2009]

• Lessons Learned from previous projects should be reviewed at the start of any project, so that we can avoid making the same mistakes twice. This is a key principle of the PRINCE2 09 project methodology.

[MS Office 2007 Upgrade 2009]

• The time spent on analysis and design proved successful in creating a robust product, and in allowing creative change. Formal specification provided a clear basis for development, and helped to keep an agreed focus. [Nursing Placement System 2002]

• Stress the importance of getting the project definition right, and fully understood and agreed by all parties.

[Video Streaming 2002]

• The project definition/project scope should be agreed within a reasonable time of project initiation – ideally within a few weeks. If this cannot be done, it could be an indication of potential problems. In this case, it would be advisable to stop the project and to re-start it only once the scope has been agreed. [Computer Asset Register 2011]

• Make sure everybody on the project team know their role and understands their responsibilities. [Video Streaming 2002]

• Clarity of project aims and scope contributed to the success of this project. This enabled the project team to focus their efforts on clear goals. [Google Applications 2012]

• User departments need to be very clear that there is a significant time commitment necessary for project team work, and for testing. [Nursing Placement System 2002]

• The institutional-wide training support should not be underestimated when an enterprise-wide system is being considered. Training and post go-live user support structures should be specified and estimated at the outset of the project. [SAP Programme 2008]

• Consider and plan for service handover from the outset of the project.

[Media Hosting Implementation 2012]


The Board/Users/Departments

• Get the right Sponsor, who really wants it to work. [eRecruitment 2010]

• Have engaged users on the Project Board – to feed in and to champion the project. Keep them involved after the project closure if possible to improve the service. The people who will run and support the service should be involved from early in the project. [eRecruitment 2010]

• Academic engagement can be achieved through the selection of enthusiastic individuals. Together with asking them to take ownership of appropriate tasks and decisions. [VLE 2010]

• Give the teams the time and autonomy to do the job. Devolve decision making as much as possible. Have regular team meetings and encourage plenty of talking to each other. [eRecruitment 2010]


• Where users [project team members] have little experience of projects and systems development there should be separate sessions on the project processes, central systems and responsibilities, as

appropriate. [Nursing Placement System 2002]

• Involve users from the outset of the project, particularly in the process of defining and agreeing the project scope and in contributing to the decision-making process. This will help in getting users to engage with the project and to feel ownership of the project. [CIS Upgrade 2011]

• Dialogue and engagement with users can be achieved through the use of informal

communication/collaboration tools like uSpace groups. Collection of user requirements needs to be carefully managed, not only to ensure that multiple survey activities are not perceived as chaotic, but also to manage expectations of the users.

• Set up a working group: A working group is the best place to have specific technical discussions. The Project Board will include members from across the university, so not everyone will be technically proficient. [Media Hosting 2010]

• Good working relationships and collaboration between members of the project team and project stakeholders contributed to the success of this project and helped the transition to the new service.

[Google Applications 2012]

• Project team members should be given training on the rudimentary aspects of working on projects and team dynamics, to prepare them for working in a project team. [SAP Programme 2008]

• It may have been better if more key users were present at the presentations to offer their support and understanding of specific areas. [Oracle GUI Conversion 2002]

• The inclusion of several frontline support staff both from CiCS and the Library was key to ensuring that the system met the users’ needs and worked for users.

[Printer Copier Management 2003]

• Clearly identify from day one of the project, who is going to take on the System Administrator & Service Manager roles and ensure that these are kept separate from the PM role. [Innovative Communications] • When introducing systems for end users the needs of the support staff and the users should be given

priority. [Printer Copier Management 2003]

• Staff time is key to a project. If departments had development time built into their staff workloads, this type of project would be more likely to be successful. [Student Finance Re-engineering 2001]


Budget and Resources

• Clarity on budget & resources is required at the outset of a project – without a budget & resource what’s the point in continuing? This is a big issue as the project could discover a good solution but then be unable to deliver it – need to manage expectations. [Business Intelligence 2009]

• Get a properly defined budget as early as possible. [eRecruitment 2010]

• If firm estimates are required before analysis (as they have usually been in order to secure resources), they are bound to be less accurate than if the process is split. It makes sense to estimate for the analysis phase, and then request resources at its completion for any proposed project continuation. [Nursing Placement System 2002]

• For large projects the budget should be established in two stages. Stage one is to cover expenditure for licence costs and the detailed specification stage. Stage two is the project budget for project completion.

[SAP Programme 2008]

• For complex and large projects the inclusion of a project accountant is necessary to assist with financial assurance. [SAP Programme 2008]


During the Project

• It is good practice to periodically review the shape/scope/direction of the project to keep the project board focussed and engaged and to ensure the project continues to work towards its fundamental goals.

[Business Intelligence 2009]

• Whilst not particularly important during the procurement phase, a greater awareness of time-scales & milestones will be critical to any success during roll-out, and a greater awareness of Project Management software might help in this. [Document Management 1 2006]

• Be open to change during the project, but talk it through properly. [eRecruitment 2010] • Have a review after each project stage. [eRecruitment 2010]

• The experience with this project reinforces the recommendation that Gateway Reviews would be helpful [with priorities and strategic direction]. [Reports Improvement 2012]

• There should be a strategic ‘gateway review’ [with SSB] at key points in all projects, and perhaps annually anyway so projects don’t get ‘lost'. [Supportworks for Service Management 2012]


• Where there is a fracturing in a project we should have an open and radical rethink, a mid-project radical review, to address underlying causes and refocus the project, or close it if appropriate.

[Supportworks for Service Management 2012]

• Don’t let the timescales slip. [Environmental Printing Review 2009]

• Users could not imagine what the screens could be like until they had seen them; by the time they had done so, it was not possible to re-design the underlying structures. Creation of sample screens would be a useful additional task to schedule to avoid this in future. [Nursing Placement System 2002]

• The rush to deliver at the end of the project did cause problems. The main lesson learned is that we should have given more time between the phases with a formal review at each stage and not to be pressured by outside companies to meet their timescales. [Oracle GUI Conversion 2002]

• The business process should always be considered for review prior to modelling the process within the system. [Document Management Implementation 2011]

• Analyse the existing processes early on, including all sides so everyone understands the whole thing and the different views. Helps get the fundamental design right. [eRecruitment 2010]

• Business process mapping and blueprinting should have in-depth representation and participation by Departments to create blueprints which are reflective of the realities in the departments. There should be ongoing revision of blueprints and plans as the project rolls out. [SAP Programme 2008]

• Analyse the existing processes early on, including all sides so everyone understands the whole thing and the different views. Helps get the fundamental design right. [eRecruitment 2010]

• The business case and benefits realisation plan should be reassessed during project implementation, especially at points of substantial project changes. [SAP Programme 2008]

• Keeping project members on task between meetings: The working group(s) should meet frequently, up to once a week, reporting to the Board.

• Routine progress meetings give structure and control to a project. Short meetings to the point are better than long rambling ones. [Student Finance Re-engineering 2001]

• If reporting structures have been agreed, those responsible for receiving the reports should ensure that they get them. [Student Finance Re-engineering 2001]

• Consider the expected benefits and baseline the existing system early on, then measure how the new one compares. [eRecruitment 2010]

• A genuine pilot phase is very useful, looking for mods needed to improve the system, training etc, not just a staged rollout. [eRecruitment 2010]


Testing, Training and Roll-out

• Create sample data and scale up to test during development so scaling problems don't emerge at the relatively late stage of user testing. [Nursing Placement System 2002]

• Design and do some bulk testing before the first training session if at all possible. [Reporting Implementation 2004]

• Testing may be more productive if users can find space away from their normal desks. [Student Finance Re-engineering 2001]

• Training needs to be carefully timed to fit in with the time users are going to start to use the system.

[Student Finance Re-engineering 2001]

• It is worth doing dry runs of major change processes such as data conversion, and doing them as early as possible to allow for corrective action. [Student Finance Re-engineering 2001]

• It may prove difficult to fit in user testing with the existing workload of users so it is important to plan testing well in advance in conjunction with the users. [CIS Upgrade 2011]

• There must be sufficient user acceptance testing of the system even if there are scheduling conflicts between performing work on refining the system versus performing the necessary user tests. The project manager needs to be provided with the responsibility and authority for resolving such conflicts. [SAP Programme 2008]

• User training should focus on the applicability to business processes as well as the “button-pushing” mechanics. [SAP Programme 2008]

• The institutional-wide training support should not be underestimated when an enterprise-wide system is being considered. Training and post go-live user support structures should be specified and estimated at the outset of the project. [SAP Programme 2008]

• Think about training and comms from the start, and start them in good time. [eRecruitment 2010] • Be aware of the limits of cascade training, and ensure enough effort is put in to make it work.



Handover and Closure

• More visibility/communication regarding what happens to outstanding tasks at project closure is required, i.e. who will take forward, what are timescales/deadlines and who will control completion? The post-implementation review provides an opportunity to review progress with outstanding tasks. The application groups need to be more visible and communicate progress/decisions. ITIL should help with outstanding operational issues. [Business Intelligence 2009]

• Emphasise that when a Closure doc is circulated to all members, this really is the end of the project!

[CRM 2– Enhancing the Student Journey 2009]

• Where team members have been working part or full time away from their normal duties there should be effort put into preparing them for closure of the project and returning to regular duties. [SAP Programme 2008]

• Post-project, the worry is that unless the service has a high-profile Champion who is willing to take ownership and sell the benefits, the service will be quietly forgotten. We recommend therefore that a working party should be set up to manage promotion of the product.

[International Institutions Database 2010]

• The Lessons Learned Review, with face-to-face meetings, is very important at the end of a conflicted project. It is particularly important that people’s views are heard where there have been major problems.

[Supportworks for Service Management 2012]


Procurement and Suppliers

• If possible, we should avoid making a major financial investment up front, until we are sure the product will provide the value we expect. [Document Management Implementation 2011]

• Make sure we have a proper understanding of our requirements before looking for a system.

[Supportworks for Service Management 2012]

• Procurement Office focus on managing and guiding the formal ‘Procurement Process’ rather than providing more substantial guidance. [Document Management 1 2006]

• A key success factor was the preparation of a detailed tender document that defined very specifically what was required and provided very clear criteria against which the competing tenders could be judged

[Printer Copier Management 2003]

• In future tenders it is recommended that the quality of the company and in particular the support it can offer to CiCS and the amount of further development it can undertake to tailor solutions to our site should be given considerable emphasis. This is of particular important when procuring systems which are not fully mature and which do not have a proven track record, such as the photocopying aspects of this project. [Printer Copier Management 2003]

• Be rigorous and objective in assessment of vendors, looking at their history and customer experience. Be very wary of unproven future functionality. [Supportworks for Service Management 2012]

• Never trust a software salesman when he offers you an alternative version on a different operating system/platform to the one you really want. [Reporting Implementation 2004]

• Get consultancy where needed, but do work in house where possible so we have the skills for maintaining & improving the system. Get good consultants. Try and know what you want, vet them, and ask for a change if it’s not working out. [eRecruitment 2010]

• When things start to go wrong with a 3rd party supplier you may have to go over the heads of the sales and

support people directly to senior management in order to get results. [Reporting Implementation 2004] • Don’t always count on support departments to fix your problems – you may know more than they do.

Legally and morally they should fix your problems, but you may waste a lot of time finding out that they can’t. [Reporting Implementation 2004]

• In the procurement process it worked well having representation by members of the services departments as well as representation from CICS, and having widespread consultation across the different

departments of University to develop the selection criteria. The use of a defined set of criteria provided objectivity to the selection process, but the documentation should provide a trail linking the requirements of the departments to the selection criteria defined. [SAP Programme 2008]

• The version of SAP provided had less functionality than the version that was demonstrated. Future negotiations of contracts for IT systems such as SAP may consider including contract terms for added consultancy support that may be needed for early versions of a particular technology. [SAP Programme 2008]

• It might be useful in a large project with staff necessarily inexperienced in the new software for the University to employ an external consultant who does not work for the vendor who has the necessary expertise to assess the quality of the vendor's consultancy and deliverables. [SAP Programme 2008]



Strategic Input & Management

• Project proposals should be clear and explicit in the objectives of the project. [CRM 2– Enhancing the Student Journey 2009]

• Stronger senior CiCS intervention and participation in the project was needed to give a clear steer on scope and direction. [CRM 2– Enhancing the Student Journey 2009]

• Need balanced and realistic project objectives so as not to initiate projects that are too

operational/reactive, too visionary, or too broad with too much variety in overall objectives. [Business Intelligence 2009]

• The scope of the project in the Project Definition was clearly unachievable. The process for PD approval should be reviewed – possibly including a more rigorous review by the Programme Board. [CRM 2– Enhancing the Student Journey 2009]

• A long term project like this needs to be phased and including stringent gateways to check on whether the project should continue, its direction and progress. The project tried to achieve too many things. A better approach would have been to agree on one area, based on a number of agreed factors, and progress that prior to expanding the scope of the project. [Document Management Implementation 2011]

• Ownership should be primarily with the business rather than the technical side, and focus on the new way of doing things not on the system itself. The business side should lead stakeholder relations/comms.

[eRecruitment 2010]

• The Project Manager needs to be able to take an objective overview. A key member of staff who is heavily involved in the day to day work of the project is not best placed to also take on the role of Project Manager and it is recommended that this scenario is not repeated for future CiCS projects. [HESA Student Record Redevelopment]

• It is not a particularly good idea to have someone project managing who also has a role/viewpoint on assessing the functionality or a serious amount of work to do in actually implementing the product. There are potential conflicts of interest, including the use of available time. [Document Management 1 2006] • Projects need to be started before the work begins so the project manager has time to properly prepare

and plan the project approach. If a situation does arise where work that is already under way turns out to require project management the PM needs senior support in this situation as he gains control of the project tasks. [MS Office 2007 Upgrade 2009]

• Input is required from the Application Groups and CiCS Programme Board to help identify and raise awareness of the links between projects. [Business Intelligence 2009]

• Potential resource issues, eg when concurrent projects are require the attentions of the same personnel, may be understood at management level, but these must be communicated to the project manager in a suitable fashion. [U-Card Replacement 2005]


General Project Management Approach

• Look at Project guidance documentation to see if there is particular documentation that may be valuable in guiding some of the processes. [Document Management 1 2006]

• Try to communicate well at all levels, up, down and across. Don’t assume that people know things.

[Document Management 1 2006]

• Have a Communication Plan and communicate effectively – keep all interested parties in the loop. [CIS Upgrade 2011]

• Good regular communication was invaluable in supporting commitment from the team (particularly SNM management) and from senior management. [Nursing Placement System 2002]

• Regular communications within the project group are essential to keep everybody up to date with progress and developments. [Video Streaming 2002]

• Speak to colleagues in CiCS who have been through similar processes. Whilst the products might be different the methodology and issues that have arisen are quite similar, and there is a lot of valuable experience to be shared. [Document Management 1 2006]

• Don’t underestimate the amount of time and effort it takes to project manage a large project. The amount of time being dedicated to it, especially in the latter stages (Proof of Concept onwards) has been

significant. [Document Management 1 2006]

• Ensure that the project manager is able to devote sufficient time to managing the project. [Media Hosting Implementation 2012]

• It is worth making sure that time is set aside for carrying out regular project management tasks. [Student Finance Re-engineering 2001]

• The project management framework offered support to the team as a whole, and allowed project momentum to be maintained despite the gap without software development. The team worked


• Formal planning was very successful in supporting the working process: it allowed team members with most knowledge in each area to be involved in the detail of tasks required. [Nursing Placement System 2002]

• The project shows clearly that the project-based approach is an effective and technique for implementing cross departmental projects. [Printer Copier Management 2003]

• We were right to keep the conversion as simple as possible because very few problems were caused by the automatic conversion. The things that caused the problems were the manually added features.

[Oracle GUI Conversion 2002] • Success factors

− Having quality people across the CiCS department with the right expertise to work with. This relates to all areas of the project, from those evaluating at a technical / practical level, to those offering informal guidance and advice on managing aspects of the project.

− Having access to these people at the right time throughout the process.

− Having a flexible timetable, and a Project Board that was only concerned with ensuring that the best product was chosen for the University’s needs and requirements.

− A lack of (unreasonable) financial / time constraint in evaluating the products.

[Document Management 1 2006]

• A key factor in the success was the high quality of the project team and their hard work. [Printer Copier Management 2003]

• Success factors:

− A project team that was suitably representative, took ownership of the project, and was prepared and able to commit time to it as and when required.

− A project management framework that supported the project and team, maintained the momentum for the project and controlled its progress without too great a level of bureaucracy.

− Clear documented and agreed statements throughout the project of what it was to deliver and when.

− Good planning and estimates of the time that tasks would take, giving clear advance warning of what was expected from team members and when.

− Initial risk assessment for the project and on-going management of risks and change as the project progressed.

− Timely and adequate testing and quality assurance of products of the project as they were delivered.

− Standards for all aspects of the project giving a professional appearance to products and engendering confidence in everyone involved.

[Student Finance Re-engineering 2001]

• Working on a project can help to stretch and develop staff. [Student Finance Re-engineering 2001] • Set realistic timescales so as not to put undue pressure on resources. [U-Card Replacement 2005] • Strict schedules and robust planning were necessary for effective project control and direction. [SAP

Programme 2008]

• Acknowledge when there is uncertainty with particular areas of work – ie those areas requiring substantial investigation or those with a steep learning curve – and reflect this uncertainty within project timescales.

[CIS Upgrade 2011]

• Do document the conversations, meetings, decisions, issues arising etc. [U-Card Replacement 2005] • Keep detailed logs of all project activity. Include: dates of meetings, to do tasks, outcomes, problems and

concerns. [Video Streaming 2002]

• Large projects with major organisational change aspects should have a key focus on business process change with the installation of the system solution being a main goal. If such an approach is adopted some of the major changes in the approach to the current project would include:

− The Organisational Change function should be structured with an accountable governance structure.

− The Organisational Change Manager role should continue for some time after software installation to facilitate embedding of the new business processes and the system solution.

− Departmental Champions with clearly defined roles should be recruited and managed to focus on the delivery of business process changes. Buy-in by departmental management will be a major critical success factor and DCs could be pivotal in this.

− Training should be focussed on the use of the system within the context of the new business processes.

[SAP Programme 2008]

• Lines of authority in the project team need to be clearly understood by project team members. [SAP Programme 2008]