An Initial model

In order to improve the software in the areas of testing and software quality, it was necessary to develop a model that could be used effectively to verify and validate the software at each stage of its development by all parties involved with the project. The first concern was to address the present problems and then to develop the solution model further, by evolving and maturing it over successive projects. The long term solution was directed at producing a framework that could be used repeatedly both in house and in other software company‟s. This initial model was directed at addressing the immediate concerns of each department in the company

Firmware and Software Development

Write a user requirement document (URD).

Document both a high level design (system specification document) and a low level design (technical specification) solution.

Document and execute Unit tests.

Maintain proper version control of builds and code.

Test

Implement a quality policy for the team

Implement a checklist for the review of each document from development to QA.

Participate at all documentation reviews for the early stages of the project and log issues in a repository raised during the reviews Produce a test strategy and plan for the project and seek peer and project approval for each test artefact.

Insist on effective configuration management of the builds that are tested and released to QA.

Manage and report the tests executed and defects detected during test execution to the project manager.

Conduct defect triage meetings to prioritise defect fixing with development

Customer Support

Review the user requirements document and proposed design solutions at an early stage of the life-cycle.

Collaborate with the development lead during use cases creation. Document acceptance tests and have these reviewed by QA before

accepting a release.

Report accurately all defects during beta or acceptance testing of beta software.

Report accurately all defects detected by customers on released software.

Testing improvements

To address the immediate problems with testing, a test process was developed which included template documents for the effective planning of testing. The test process documents were designed in addition to new process documents from the development departments. Each document was to be peer reviewed by the author‟s colleagues before being subjected to an interdepartmental review. Following on from the review, any open topics raised at the review would be followed up by the author as action points. The author would follow up on those action points with a second review or send out an updated document with amendments as appropriate.

The initial documents from development would be a user requirements document (URD) and system specifications document (SDS). The URD would record what was required from a software solution and the SDS how that solution was going to be implemented.

The information contained in the URD, SDS would be used to create the test strategy document. The test strategy documents purpose would be to describe at a high level what the test approach to the project will be. The test strategy would also be used as a matrix to map user requirements and design specification points to tests. The test strategy would also allow the test lead to make preparations for a test environment and to source requirements for adequate test data.

The test plan would be a more detailed explanation of the testing approach and act as a test schedule for the project. It would expand on each of the areas of the test strategy but also set out in more detail each of the points from the URD and SDS in terms of what will be tested and when. The test data and test environment details would also be documented in advance of their configuration in the test plan.

The test execution process would follow the test planning process. The first part of this process would be a formal handover of builds from development to test. This handover would maintain the requirement for version control of both code and builds. The details of defect fixes would form part of this handover form. This would assist with the defect management and regression testing and maintain the test status of different builds. Bug fix reports were to be completed and compiled together and form part of the handover of builds to QA. This configuration management practice assisted with the quality assurance of individual builds. The test results would be documented with defect states so that progress reports could be compiled with an assessment of software quality. Metrics for test case completion and defects per component and build would be recorded to assist with the identification of the root cause of defects. This information would be factored in for quality improvement in successive projects.

Following QA test execution completion and signoff, the application was handed to the customer support department to conduct user acceptance tests either on site with approved Beta customers and or in house.

The processes were documented and included in the company‟s‟ quality system. The company documentation for this process can be found in appendix A. A diagram of the test process and test execution procedures is depicted below in figure 5.9.

Fig 5.9 improved test and QA process

The company‟s quality system was changed to reflect the new test process and supporting development documentation. The documents that were changed to the company quality system are included in the appendix A. The documents are listed below:

ECR – 0100 Testing Research Plan Procedure 0029 writing test documents Procedure 0056 software testing procedure Work instruction 0032 test script creation

Work instruction 0005 dealing with an incident in released software Work instruction 0081 use of Bugzilla for defect tracking

Form 0105 software handover form Form 0123 firmware handover form Form 0127 SW test report form

5.4 Summary

In this chapter the industry sector that the company operates in was explored. Elements of the company, specifically the R&D department, were described in full. The function of the R&D teams and the company products were described in terms of the BMS architecture. The architecture and operation was explained in depth. In relation to the BMS system, the fundamental of the engineering application was explained and its role in the BMS system put into context.

The fact that the company was experiencing quality problems was mentioned. Investigations into the quality problem were conducted in an assessment. The assessment findings were explained in detail. Details of the findings included feedback from customers, internal departmental and company quality process audits. The root causes of the assessment findings was compiled into an engineering change request / report which outlined a proposed solution to the problem. The proposed solution was described and depicted in a graphical process. The process included testing and quality assurance practice improvements. These improvements were to be implemented and evaluated over forthcoming company projects. This implementation is described in the next chapter.