CloudSizzle : Quality Assurance Plan
Quality Assurance Plan
• General info• Changelog • 1. Introduction
• 2. Quality goals and risks • 3. Quality Assurance practices • ° 3.1 Testing levels
° 3.2 Testing
° - 3.2.1 Test case based testing
- 3.2.2 Exploratory testing
- 3.2.3 Automated unit testing
- 3.2.4 Performance and load testing
- 3.2.5 Regression testing
- 3.2.6 Test suites
- 3.2.7 Pass and fail criteria for test cases
- 3.2.8 Definition of done
° 3.3 Other quality assurance practices ° - 3.3.1 Code and document reviews
- 3.3.2 Coding and code documentation standards
- 3.3.3 Static code analysis methods
- 3.3.4 Defect tracking
- 3.3.5 Pair programming
- 3.3.6 Refactoring
- 3.3.7 Unit test and code coverage
- 3.3.8 Quality checklist
- 3.3.9 Collecting feedback from the customer
- 3.3.10 Automated acceptance testing ° 3.4 Tracing quality goals to QA practices • 4. Schedule
• ° 4.1 Iteration 1 schedule regarding QA ° 4.2 Iteration 2 schedule regarding QA • 5. Resources, tools and testing environments • ° 5.1 Quality assurance roles and responsibilities
° 5.2 Tools
° 5.3 Testing environments and test data ° - 5.3.1 Testing environments
- 5.3.2 Test data • 6. Deliverables
• ° 6.1 Quality assurance plan ° 6.2 Quality assurance report ° 6.3 Test cases
° 6.4 Test session charters ° 6.4 Defect reports
° 6.5 Summary of peer testing results ° 6.6 Quality checklist
• 7. Evaluation and feedback • References
General info
Responsible for document Jari Pääkkö
Changelog
Version Date Change
description
Author Reviewer
0.1 27.10.2009 Created the initial template for the QA plan
Jari Pääkkö Ville Koivunen
0.2 27.10.2009 Updated the introduction and quality goals
Jari Pääkkö Ville Koivunen
0.3 27.10.2009 Updated the quality assurance practices and attached the V-model of testing picture
Jari Pääkkö Ville Koivunen
0.4 27.10.2009 Updated the section on deliverables
Jari Pääkkö Ville Koivunen
0.5 27.10.2009 Updated the resources, tools and environments
Jari Pääkkö Ville Koivunen
0.6 27.10.2009 Updated the evaluation and feedback section
Jari Pääkkö Ville Koivunen
0.7 28.10.2009 Updated the QA practices and testing section
Jari Pääkkö Ville Koivunen
0.8 28.10.2009 Updated the schedule
Jari Pääkkö Ville Koivunen
0.9 28.10.2009 Correcting typos and making small modifications where needed
Jari Pääkkö Ville Koivunen
1.0 28.10.2009 Changed the status of the document to proposal
Jari Pääkkö Ville Koivunen
1. Introduction
This document describes the most important quality goals, used quality assurance practices, testing environments, produced QA deliverables and the evaluation of the used QA practices for the CloudSizzle system. A brief schedule will also be presented that describes when the different QA practices are performed in the project. This document will be updated during the project as feedback from the customer regarding quality assurance is received and as new information from the performed QA practices are gained. Table 1.1 displays the intended audience of this document.
It is worth to note that one of the business goals of the CloudSizzle system is to get an understanding of the Smart-M3 RDF Store. Thus, the system will be developed as a skeleton system that can be used to
validate the architecture and developed further, if the customer sees it necessary. As such, the system will not be developed as a production quality service in the beginning.
Table 1.1. Intended audience of this document.
Group of the readers Reasons for reading
The customer To be able to see that the planned quality goals and practices are sufficient and meets the needs of the customer. To be able to give feedback about the quality goals and performed quality assurance practices.
System architect To use this document as a basis for the
architecture description of the system, especially for the most important quality properties of the system.
Project manager To be able to see the status of the quality goals, number of defects and performed quality assurance practices.
Quality assurance manager To use this document as a way to communicate the planned quality goals and practices.
Developers To understand the quality goals and communicate the quality assurance practices.
Testers To learn about the quality goals and quality assurance practices.
Mentor To get an understanding of the quality goals and quality assurance practices of the project and the status of these goals and practices.
Peer group To give an understanding of how the peer evaluation is performed
2. Quality goals and risks
Table 2.1 lists the most important quality goals of the system and deliverables as well as their verification criteria. These goals will and their status wil be updated throughout the project in order to reflect the most important quality goals of the system. The quality goals and their priority is updated in the iteration planning meetings by the customer.
Table 2.1. List of quality goals
ID Quality goal Description Verification Priority
QG-1 Maintainability The system shall be easy to maintain so that it is possible to test, fix defects, make changes and improve the code without huge amounts of effort. • All methods and functions must have comments • The system shall follow the DRY principle • The code complexity of different
parts of the system shall be at maximum between 7 and 9 • It must be possible to write unit tests to all critical functionality QG-2 Extensibility The system
shall be easy to extend so that new features and functionality can be implemented. • The system architecture must be modular • There shall not exist too many dependencies between classes • The interfaces must be well defined and documented in the code QG-3 Documentation The documentation of code, user manuals and other documents must be done well. • The documents should be type checked so that they do not contain too many type errors • The text must be understandable by a person that has a basic understanding of software engineering and computer science • The documents shall follow the DRY principle QG-4 Security and privacy
The system must be secure so that it prevents information leak, information loss and illegal use.
• The system must prevent the most usual web attacks such as SQL injections
• Best practices according to security must be followed • Users' passwords to other services such as Noppa and Oodi must be securely handled so that they are not compromised to third parties • Users shall be able to change their privacy settings QG-5 Internationalization The system
must support internationalization without the need to modify the architecture. • The system must support language files that enable the customer to easily add new languages • The system must support Finnish and English QG-6 Performance and scalability The overall performance of the system must be sufficient and the system must be scalable without the need to modify the architecture. • The system must perform it's most important requirements within the time requirements defined in the architecture description • The system must be scalable to 20.000 users without the need to modify the architecture QG-7 Interoperability The system must
be interoperable so that it is easy
• It must be
possible to connect new
to link more systems to it, if needed. services to the system without the need to modify the architecture QG-8 Completeness The degree to
which the system possesses the necessary and sufficient functions to satisfy the user needs. • The system must satisfy the most important requirements defined and verified by the customer at the moment of development QG-9 Correctness The degree to
which the system performs its required functions. • All critical defects must be fixed at the end of the project so that the system can perform its required functions
Table 2.2 list some of the quality risks for the system. This list will be updated throughout the project.
Table 2.2. List of quality risks
ID Quality risk
QR-1 Group members are not familiar with Python QR-2 Code documentation is not sufficient
QR-3 The performance of the system suffers from the many components
QR-4 The system does not satisfy the customer needs QR-5 The system is difficult to maintain
3. Quality Assurance practices
This chapter presents the quality assurance practices that are performed during the project.
3.1 Testing levels
During the project various types of testing will be performed in order to understand and improve the overall quality of the system. Testing will be performed at different levels as Figure 1 shows.
Unit testing will be the first stage in the testing process. Testing at this stage is done with white-box methods by testing small units of the system. A unit is the smallest possible testable software component, that is in this case classes and their methods. Faults discovered at this stage are usually much cheaper to fix than in the later stages or when the system has already been delivered to the customer.
Integration testing is the next phase in the testing process. The purpose of integration testing is to discover faults between the interfaces of different components of the system. Continuous integration and automated unit tests are somewhat helpful for integration testing. However, integration testing is usually performed with black-box testing methods such as equivalence partitioning, boundary value analysis and pairwise testing.
System testing is the third stage of testing in the V-model of testing. Its purpose is to test the integrated system to evaluate it's compliance with the requirements. It is done using black-box testing methods and requires no knowledge of the inner design of the system. Some types of system testing are for example performance testing, security testing, exploratory testing and regression testing.
Acceptance testing is the final phase of testing. It determines whether the system satisfies the requirements that have been specified in the requirements document. It is also done using black-box testing methods. Testing in this phase can be distinguished by testing that is done by the project team and testing that is done by customers. The project team can use for example the Selenium testing framework as a method of acceptance testing. The acceptance testing that is done by the customer on the other hand is done in a real environment where the system will be used and the customer determines whether the requirements are satisfied or not.
Figure 3.1. The V-model of testing.
3.2 Testing
This chapter describes the different testing methods that will be used during the project. Generally testing activities will be carried out throughout the project.
3.2.1 Test case based testing
Test case based testing is a form of functional testing. During the project test cases will be written for all use cases that will be developed in the iterations. These test cases will be documented in a spreadsheet in Google docs. Each test case will have a unique identifier, a short name, a description, related requirements or use cases, a priority, input values, steps, expected results and notes. The test logs that are documented on the same spreadsheet consist of a date, name of tester, test environment, list of executed tests, their status (passed or failed) and defect identifiers.
The testing of test cases is performed both manually and automatically. Manual testing is performed on functionality that is of low importance, while automatic testing is performed using the Selenium testing framework on requirements that are of high importance and are changing frequently. In addition, some of the pairwise test cases will be generated with a tool called Allpairs.
3.2.2 Exploratory testing
Exploratory testing is a form of functional testing. The distinction between exploratory testing and other types of functional testing is that testing is designed and executed at the same time. Exploratory testing and finding defects with the help of it is very much dependent on the experience of tester.
During the project Session-based Test Management will be used as the exploratory testing approach. These sessions will be conducted both in the I1 and I2 iterations in the system testing phase of functionality. A session will last approximately 60-90 minutes and during the session the tester will produce a test charter as is described in section 6.4 Test session charters. The tester can use mindmapping software during the session to help quickly document the different decisions and found defects of the session.
3.2.3 Automated unit testing
When delevopers implement classes and methods for these classes, the developers are also responsible for writing unit tests on the implemented functionality. These unit tests will be grouped together as automated test suites that are executed by the continuous integration system on a regular basis and when a developer commits his work to the repository. The unit tests will be written with the unittest (sometimes referred to as PyUnit) library that is included with Python.
3.2.4 Performance and load testing
Performance testing is carried out on some of the most critical functionality. A tool called pyUnitPerf is used for measuring the performance and scalability of functionality contained within existing PyUnit tests. JMeter is another tool that can be used during the project to measure load testing of the system.
3.2.5 Regression testing
Regression testing is performed during the project with the help of the continuous integration tool and manual testing. The continuous integration tool executes the automated unit tests and can be used to easily discover, if something has broken as a results of a change. Manual testing may also be performed on some critical functionality that has changed in order to ensure that everything is working as specified in the requirements document.
3.2.6 Test suites
The different test cases will be grouped to test suites in order to ease the management of tests. The tests will generally be grouped by functionality, but depending on the situation another approach can also be used. The different test suites will also be prioritized according to customer value and the criticality or complexity of the component.
3.2.7 Pass and fail criteria for test cases
A test has passed, if the actual result is the same as the expected result. If the expected result is not the same as the actual result defined in the test case, then the test has failed.
Test suites have passed, if all the tests contained in the test suite have passed the tests. In other words, all tests inside test suites have to pass in order for the test suite to pass.
3.2.8 Definition of done
All requirements that are implemented, but not tested are not considered ready. Only the requirements that have been tested and have passed all the tests that are related to the requirement can be
3.3 Other quality assurance practices
This section describes other quality assurance practices that will be performed during the project.
3.3.1 Code and document reviews
Code reviews
Code reviews will be organized twice in every iteration. The purpose of code reviews is to communicate best practices regarding programming and find possible coding standard violations and defects in the code. Thus, it functions both as a way to educate programmers and improve the quality of the system. The code reviews will be held in the group meeting room in the CS building. The group members, whose code is being reviewed must attend the code review session in order to present their code. The architect and other developers are also required to attend the code review and their role is to function as reviewers. The reviewers will receive the code that is reviewed before the meeting so that they can prepare in time. The group members, who are attending the code review will evaluate and review the code based on the quality checklist that is described in more detail in chapter 3.2.8.
The most critical parts of the system will be reviewed in the code reviews. Participants may also suggest other noteworthy or otherwise important parts of the system for subjects of code reviews. These
suggestions must, however, be sent in advance to the other participants.
Document reviews
During the iterations the project team will have internal deadlines for the deliverables in order to ensure that there is enough time to write them well. Document reviews will be carried out before these internal deadlines. The process of document reviews is similar to code reviews above.
3.3.2 Coding and code documentation standards
As the system is developed with the Python programming language, the coding standard that is followed is the Python Enhancement Proposal 8: Style Guide for Python Code (http://www.python.org/dev/ peps/pep-0008/). It can be regarded as the official coding standard for Python and it gives a detailed description on
how to write the code.
The Django MVC framework that is used in the project also has a coding convention that is based on PEP 8 (http://docs.djangoproject.com/en/dev/internals/contributing/#coding-style). Accordingly, when writing code in Django, the coding conventions for the framework must be used.
Furthermore, there is a Python Enhancement Proposal that describes some best practices when programming with Python. PEP 20: The Zen of Python (http://www.python.org/dev/peps/pep-0020/) describes these best practices as aphorisms.
The code needs to be documented in order to improve maintainability and refactoring of the code. Thus, it is important to follow a standard for code documentation. The Python Enhancement Proposal 275: Docstring Conventions (http://www.python.org/dev/peps/pep-0257/) will be used as the standard for writing docstring in Python. One advantage of using a standard for code documentation is that it is then possible to generate the documentation from the source code. In this case the Python module pydoc will be used to generate the documentation.
3.3.3 Static code analysis methods
Code review is a form of static code analysis. However, this chapter will describe how automated static code analysis methods will be used to find coding standard violations and bugs in the system.
Pylint (http://www.logilab.org/857/) is a tool that can be used to find bugs and signs of poor quality by examining the source code against coding standards. Pylint can be used together with a continuous integration tool to generate reports about code quality. As Pylint is bundled with Pydev, a plugin for Eclipse that enables users to use Eclipse for programming Python, the developers can easily check their source code for problems before commiting it to the repository.
RATS (Rough Auditing Tool for Security, http://www.fortify.com/security-resources/rats.jsp) is a tool that can be used to scan Python source code in order to find security issues. This tool should be used
to complement code reviews and other manual static code analysis methods to find issues related to security.
There is also one tool called pygenie ( http://www.traceback.org/2008/03/31/measuring-cyclomatic-complexity-of-python-code/) that measures the cyclomatic complexity of Python code. Complex programs are generally more difficult to unit test and maintain. Thus, this tool can be useful for measuring how complex the code in the system is and where it should be improved. If the code complexity increases to more that seven, there is a need for refactoring the code. Another tool called PyMetrics (http:// sourceforge.net/projects/pymetrics/) also calculates the cyclomatic complexity, LoC and the percent of comments in the code. At the moment of writing these tools have not been tested by the members of the group. However, most likely only one of these tools will be used in the project.
Snakefood (http://furius.ca/snakefood/) is a useful tool that can be used to generate dependency graphs from Python code. Controlling the depencencies between various parts of code is good way to increase the reusability of code. In other words, the less dependencies a part of code has, the greater the possibility to reuse that code in the future.
CloneDigger (http://clonedigger.sourceforge.net/) is a tool that will be used to find similar or duplicate code in Python programs. This tool can be very useful for measuring how well the DRY principle is followed.
3.3.4 Defect tracking
Defect tracking is an important part of quality assurance. It also provides useful metrics for evaluating the quality of the system. In the project Trac is used for defect tracking as it is also used for project management. All of the stakeholders will be provided access to Trac so that they can report and monitor defects and change requests.
Stakeholders can report defects, change requests and other issues as tickets in Trac. The user can choose the type of ticket from the available options. These tickets are then reviewed in the sprint planning meeting according to their severity and priority. In the sprint planning meeting the stakeholders also decide, which tickets will be implemented in the upcoming sprint. The QA manager is responsible for the testing of defects and implementation of changes and delegates these tasks to testers and developers as necessary.
The defects are assigned to developers based on the defects. Table 3.1 lists the fields used in defect reports:
Table 3.1. List of fields used in defect reports
Field Description
ID A unique identifier for the defect
Name A short name for the defect that helps developers to quickly distinguish the defect from other defects.
Description A more detailed description about the defect and how it affects the system.
Test case ID The unique identifier of the test case that revealed the defect.
Reproducibility The field describes the reproducibility of the defect. The options are as follows: always, sometimes, random, have not tried, unable to reproduce.
Steps to reproduce A description of steps that help to reproduce the defect.
Actual result The result that was achieved with the step-by-step instructions.
Expected result The results that was expected when performing the step-by-step instructions.
Component / module The component or module that the defect concerns.
Severity / impact The severity of the defect.
Priority The priority of the defect. The higher the priority the sooner the defect should be fixed.
Time The time when the defect was submitted. Reporter The reporter of the defect.
Build The build version, in which the defect was found. Environment The environment that was used when the defect
was found.
Related requirements / use cases A reference to a related requirement / use case that describes how the feature should work. Related defects A reference to other similar or related defects. Related test cases A reference to test cases that are not working,
because of this defect.
Attachments Attachments that can help in identifying or fixing the defect. For example screenshots, logs or videos.
Additional information If there is some other information, for which there is no field.
3.3.5 Pair programming
Pair programming is a lightweight version of code reviews. It is useful for distributing knowledge between developers and to improve the quality of the system. Generally, senior or more experienced developers should be paired with less experienced developers. This way junior developers are able to quickly learn the best practices and to adapt to new technologies. In order to increase the knowledge distribution of the system within the group, the pairs will be changed regularly.
In pair programming the other developer works as the so called driver, while the other developer is the navigator. The driver is the person, who writes the code. The navigator on the other hand reviews each line of code as it is typed. The two developers should switch roles frequently, for example every 30 minutes.
Because of the nature of the project, it may be difficult to organize sessions, in which several developers can attend at the same time. Nonetheless, pair programming will be used as much as possible.
Developers may also discuss with other developers on their own in order to organize a small pair
programming session with another developer. If possible, remote pair programming sessions may also be organized.
3.3.6 Refactoring
Refactoring is an important part in the evolution of a system in order to improve it's internal quality attributes. Refactoring occurs by modifying the internal structures of a system without changing the external behaviour of it. The main advantages of code refactoring are the improved readability,
maintainability, performance and extensibility that is gained with it. As the development process is iterative, it is important to refactor the code regularly.
Refactoring is something that all of the developers are responsible for. The needs for refactoring a part of code may be discovered at code reviews, pair programming sessions or by some other automated static code analysis tool. Automatic unit tests ensure that refactoring does not break existing functionality.
3.3.7 Unit test and code coverage
There are some code coverage analysis tools that can be used to measure, which parts of programs are executed while running them. This can be useful, when writing unit tests and understanding what parts of programs are really tested. Code that is not covered by unit tests is not tested.
3.3.8 Quality checklist
A quality checklist will be available for the developers. This quality checklist lists different best practices and tests that the code has to conform to. This checklist can be used when reviewing code in code reviews or before commiting changes to the repository.
3.3.9 Collecting feedback from the customer
The development process will use an iterative development model. The project will be split up into two implementation iterations and these iterations will further be split up into shorter sprints. At the beginning of the iterations the customer will prioritize the quality goals. Also, at the end of each sprint an increment of functionality should be ready. This functionality can then be demonstrated to the customer. In addition, at the end of each iteration there will be an iteration demo at which some functionality will be demonstrated. This demonstration serves as an event where the customer can give feedback on the quality goals and implemented deliverables. Moreover, the customer or some customer representative can be invited to code reviews in order to get feedback about the code quality and documentation.
3.3.10 Automated acceptance testing
The Selenium testing framework will be used for automated acceptance testing. It can also be used to test the web application with modern browsers, eliminating the need to test that manually.
3.4 Tracing quality goals to QA practices
This section describes how the quality goals and risk trace back to the used quality practices.
Table 3.2. Summary of QA practices and quality goals
Practice QG-1 QG-2 QG-3 QG-4 QG-5 QG-6 QG-7 QG-8 QG-9 Test case based testing x Exploratory testing x x Automated unit testing x Performance and load testing x
Regression testing x x Code reviews x x x x x Document reviews x Coding and code documentation standards x x Static code analysis methods x x x x Defect tracking x Pair programming Refactoringx Unit test coverage x Quality checklist x Collecting feedback from the customer x Automated acceptance testing x
Table 3.3. Summary of QA practices and quality risks
Practice QR-1 QR-2 QR-3 QR-4 QR-5 Test case based testing Exploratory testing Automated unit testing Performance and load testing x
Regression testing Code reviews x x x x Document reviews Coding and code documentation standards Static code analysis methods x x Defect tracking Pair programming x Refactoring x Unit test coverage Quality checklist x Collecting feedback from the customer x Automated acceptance testing x
4. Schedule
Quality assurance will be carried out troughout the project. The schedules will also be updated as soon as more information is available regarding the iteration meetings etc.
4.1 Iteration 1 schedule regarding QA
Date Time Event Description Participants /
responsibles Status 28.10.2009 13.00 Deadline: QA plan Jari Pääkkö Done Unscheduled events
Event Description Participants Code review session 1 The first code review in the
iteration
Code review session 2 The second code review in the iteration
All group members
Document review session 1 The first document review session in the iteration
All group members
Document review session 2 The second document review session in the iteration
All group members
Exploratory testing sessions Exploratory testing sessions Testers Writing the test cases for use
cases
Writing test cases for the requirements that are implemented in the iteration
Testers
Writing Selenium test cases Writing Selenium test cases that can be used for automatic acceptance testing
Testers
Writing QA report Writing the QA report that is delivered at the end of the iteration
Jari Pääkkö
4.2 Iteration 2 schedule regarding QA
This chapter will be updated at the beginning of I2 iteration.
5. Resources, tools and testing environments
This chapter describes some of the resources that are needed for quality assurance as well as roles and responsibilities regarding quality assurance and different tools that are used for quality assurance.
5.1 Quality assurance roles and responsibilities
Table 5.1. List of roles and responsibilities regarding QA
Role Responsibilities
QA manager • Responsible for all quality assurance
documents and updating them
• Assigning quality assurance related tasks to
other group members
• Present the quality report at the iteration
demo
• Oversee that the quality assurance practices
are performed
Project manager • To steer the project according to the quality
metrics
Software architect • To take into consideration the most
important quality goals in the architecture
• To see that the most important parts of the
system are tested sufficiently Testers • Write test cases
• Perform exploratory testing and other test
Developers • Follow the quality assurance practices
written in the QA plan
• Write unit tests • Report found defects • Fix defects
Peer group • Perform testing on the system
5.2 Tools
Table 5.2. List of tools needed for QA
Name Description
unittest (PyUnit) A unit testing framework for Python.
Pylint A source code analyzer that looks for defects and poor signs of quality.
Clone Digger A tool that can be used to find similar or duplicate code in Python programs.
Trac A project management and defect tracking system.
Bitten A continuous integration plugin for Trac. RATS RATS is a tool that can be used to scan Python
source code in order to find security issues. Snakefood Snakefood is a tool that can be used to generate
dependency graphs from Python code.
PyMetrics A tool that calculates the cyclomatic complexity and lines of code in source code.
pyUnitPerf A tool that is used for measuring the performance and scalability of functionality contained within existing PyUnit tests.
Selenium A software testing framework.
JMeter Load testing tool that can be used to measure the performance of web applications.
Google docs A tool that enables collaborative editing of documents and spreadsheets.
Freemind A mindmapping tool.
Allpairs A tool that can be used to generate pairwise test cases given some predefined inputs.
5.3 Testing environments and test data
5.3.1 Testing environments
The continuous integration and automated tests will be executed on the server provided by the customer. The server runs Fedora Core 11 and the continuous integration software is complemented with different software metrics tools. The developers will also test the functionality on their own computers before committing the changes to the repository.
The Noppa portal provides a test system that can be used during the project to test integration to that service. Developers are able to login with their weblogin password that they have received from HUT IT-services. Thus, there is no need for test accounts for Noppa. There are currently also talks about getting access to a test version of WebOodi or to get test accounts for the live version.
5.3.2 Test data
Test data for the Noppa portal is available in the test version. Thus, it is not necessary to write test data for that service. The Django web framework that is used also contains a fixture component that can be used to generate test data.
6. Deliverables
This chapter describes the deliverables that are produced as a result of quality assurance.
6.1 Quality assurance plan
The quality assurance plan describes the quality goals and quality assurance practices used in the project. It is updated throughout the project in order to display the current situation of the system quality.
6.2 Quality assurance report
The quality assurance report is used to communicate the status of the project's quality goals as well as the quality of different parts of the system to all stakeholders. Different types of quality metrics will be used to will be used to assess the status of quality. A quality assurance report will be produced at the end of both I1 and I2 iterations.
The report will describe what quality assurance practices have been used and how they correspond to the planned quality assurance practices. The QA reports will also describe the quality status of different parts of the system with the help of a quality dashboard. In addition, the quality goals and their status is reported as well as relevant quality metrics as described above.
6.3 Test cases
Test cases will be documented in a spreadsheet in Google docs as well as test logs related to the test cases.
6.4 Test session charters
As a result of exploratory testing test session charters will be produced. The test session charters describe what is tested, the goals of testing, the approach and exploration logs. Some test session charters will also be produced as a results of peer testing for the peer group.
6.4 Defect reports
Trac is used to for tracking defect reports and change requests. A more detailed description on defect tracking can be found in section 3.2.4 Defect tracking.
6.5 Summary of peer testing results
During the project peer testing will be performed for the peer group. The results of peer test must be written in a peer testing document that describes the activities and findings of peer testing. The peer
testing group will also produce a peer testing summary that describes the results of their testing of the CloudSizzle system.
6.6 Quality checklist
A general checklist that can be used in code reviews to review the code against.
7. Evaluation and feedback
The information gained from using QA practices is used to steer the project to the right direction according to the most important quality goals. The quality status of the system will be evaluated after each sprint and also in the iteration demo at the end of the iterations. The quality status will then be presented in the quality report that is delivered as well as in the iteration demo.
The interaction with the customer is important in order to achieve the quality goals. Thus, the quality goals along with the metrics will presented to the customer during the iteration in order to get feedback. The quality practices will be evaluated according to how they manage to accomplish the quality goals. If the defined QA practices are not sufficient or do not accomplish the quality goals, the QA practices may have to be changed.
