Real Estate Agents Authority
Core Business System Upgrade
(CBSU) Project
Quality Assurance Review
Author: Stuart Judge Date: March 2013 Version 2.0
Contents
Real Estate Agents Authority
1
CBSU Project Review ... 1
Introduction ... 1
History ... 2
Observations ... 3
Real Estate Agents Authority - CBSU QA Review Page 1
Real Estate Agents Authority
CBSU Project Review
Introduction
During 2012 the REAA commissioned Gemtech to conduct an independent project review of the Core Business Systems Upgrade (CBSU) project since its inception in 2011. Their investigation was thorough and a subsequent report of findings was produced dated 1 February 2013. The Gemtech report noted:-
• Following a contestable RFP process, Datacom was the selected vendor to
develop a new core business IT system
• The initial Datacom estimate for this work was $654K based on a document
describing the high level functional requirements for the new system
• REAA set an expectation with the Board of a capital budget of $1.2M and
that at the time of the review the project was on target to meet a revised figure of approximately $1.8M
• The first phase of the project was scheduled for completion by the end of
February 2013
• Further analysis and discussion is required to define and quantify the actual
requirements for the remaining phases two and three of this project.
The report was highly critical of many key aspects of the project and also noted that the projects objectives had not been delivered in the agreed time and cost. Many of these shortcomings arise from poorly documented requirements within the original RFP and the AGILE development methodology employed by Datacom. Key components for managing a project of this nature were found to be missing from the operations of both REAA and Datacom.
Gemtech made many valuable recommendations to the Board including:-
• Reworking the business requirements documentation for all remaining
development to a higher standard than previously used for the initial phase
• Obtaining external Quality Assurance advice for remaining phases of the
project to ensure best practice is adopted and followed.
REAA subsequently froze any further development work on the project until a review of the business requirements documentation for the remaining phases had been completed.
In March 2013 Jarin was contracted to provide a Quality Assurance assessment of the project in its current form and recommendations for the project proceeding through to completion.
CBSU Project Review
History
The original REAA computer system developed in 2009, known as REGIE, was not capable of delivering modern self-service facilities via the internet for both the industry and the public. The CBSU project is intended to heavily automate the administrative functions of the organisation and provide a fast and efficient means for the industry to interact with REAA.
The original RFP functional requirements attempted to introduce this desired automation by describing many self-service business functions such as applying and renewing a licence. The RFP response from Datacom makes it quite clear that they understood this intention. However, the specification is loosely worded in parts and allows interpretation as a system that either partially automates an existing manual process, fully automates the process or a mixture of both.
For instance, the original intention of the specification allowed for Licensees to renew and pay their licences on line via the web site. Unfortunately the specification goes on to say "or by downloading a form". The Datacom RFP response provides many examples of how a fully automated on-line renewal system would work although that functionality is yet to be built even though the cost to date for the complete system has almost doubled from the original RFP estimates. The current system of renewal still uses downloaded PDF forms from the REGIE era and requires up to 15 REAA personnel to manually rekey that information for nearly three months of every year.
During 2012 Datacom ran a series of workshops with REAA personnel to detail how the original objectives of the RFP would be met. A series of more detailed specifications were developed that indicated further development costs in excess of $600K were still required to meet those objectives.
Late 2012 REAA accepted the GEMTECH recommendation to bring the results from the Datacom workshops in house and to regenerate its own requirements specification for all remaining development from that source data. That
specification for all remaining development within the CBSU project has a completion date of 8 April 2013.
Real Estate Agents Authority - CBSU QA Review Page 3
CBSU Project Review
Observations
The following observations are made following discussions with REAA and Datacom personnel:-
a) The single largest risk factor for the CBSU project since the RFP was issued has been the lack of detail in the requirements specification. The
specifications were at a very high level with the actual detail to be defined at a later date. Specifications for a project of this size are required to be complete in every detail, including a full working prototype of all screen functionality. REAA has recently accepted the GEMTECH recommendation for using prototypes as part of the revised specification.
b) The quality of the new draft specifications is a significant improvement on those used for phase one. However, the revised completion date of 8 April 2013 for all remaining specifications and prototypes to the level required for a successful project outcome is unlikely to be met.
c) The specifications and prototypes viewed during this review do not take full advantage of technology available to roll out a modern self-service system with highly automated back office support. Development of IT systems to these specifications will not be value for money and will increase, rather than decrease, the operating costs of the organisation.
d) Those involved in creating the new specifications have not embraced the "totally electronic" model. For instance, the specifications still allow Licensees to download blank PDF forms, complete those forms manually, attach a cheque and return both items to REAA via surface mail. This functionality is unnecessary, is expensive to develop and will increase future operational cost for REAA.
e) There appears to be a "disconnect" between the Board and those working on the specifications as to required outcomes of CBSU project.
f) The system developed to date at Datacom appears to use industry standard components and is solid technology upon which to conduct further
CBSU Project Review
Recommendations
The following recommendations are made:-
1. The Board and Chief Executive should review and set firm strategic objectives to be met by the project in parallel with the remaining specifications being completed. These objectives should include the adoption of a total electronic process and the elimination of all manual documents and payment methods. All interaction with external users should be on line and conducted exclusively within an electronic format with
cheques, paper and pdf forms totally eliminated. A policy of only accepting electronic payment ought to be adopted.
2. A Board appointed CBSU Steering Committee should be formed, including user representation along with project managers from both REAA and Datacom. The CBSU Steering Committee should be chaired by the Chief Executive and be responsible for overseeing all software development through to completion and roll out. The Board should appoint an
independent Quality Assurance consultant to the CBSU Steering Committee and the chair should provide regular project status and expenditure reports to the Board or its existing sub committee.
3. Experienced project managers at both Datacom and REAA need to be appointed and for each to develop their own project plans that are synchronised with each other.
4. The existing specifications and prototypes for all components of the system need to be completed and reworked by REAA to meet the abovementioned strategic objectives.
5. Internal users at REAA need to be appointed to be responsible for
completing and signing off their areas of the specification and prototype in conjunction with the Chief Executive. Those same individuals, or a
dedicated User Acceptance person, need to be responsible for ensuring all software delivered by Datacom meets those specifications.
6. As an organisation REAA must be prepared to sign off and freeze all specifications for the duration of the development phase. Any subsequent requests for change should be included in a separate register and reviewed for future development at a later date. Any change request deemed to be a business "showstopper" should be referred for Board approval along with the impact to the project in terms of risk, cost and delivery date.
Real Estate Agents Authority - CBSU QA Review Page 5
CBSU Project Review
Recommendations (contd)
7. The entire specification for all development should be submitted to Datacom along with a development priority set by REAA. Priority should be given to new licence applications as the first module followed by renewals as the second. Datacom and REAA should jointly reduce all development phases down to 30 to 60 day modules and for Datacom to provide both fixed cost and elapsed development times for each module. A project plan, budget and business case for all future development should then be presented to the Board for approval.
8. A fixed price contract should be entered into with Datacom that requires each module to be completed strictly in accordance with the project plan. Contract penalties should apply and be strictly enforced for late delivery of any module or any software produced that is either deficient or fails to meet the requirements specification. REAA should only authorise the
commencement of each module once the previous module has been successfully delivered in full.
9. A small group of external users with high transaction volumes and complex requirements should be identified and invited to view the prototypes before and after development. That same group should be invited to pilot each module for a short period prior to it being released to the wider public. 10.A communication and public release project plan should be developed to roll
