Software Quality Engineering
BS(SE)-VI
Dr. Assad Abbas
Department of Computer Science
Topics
Problem
n “How exactly do you test your requirements?”
n The best way to solve this problem is to introduce a
requirements analysis stage that has to be done before coding starts.
n No one knows a product as well as a tester who works
Requirements Quality Assurance (QA)
n
Goal:
5
Quality assurance is aimed at
checking
that the items
specified in a Requirements Document (RD) meet the
desired qualities attributes
Completeness, consistency, adequacy
n
Why is Requirements QA important?
5
Because
cost of requirements failures
is very high
Common Requirement Defects…
Making Requirements Measurable
n Define ‘fit criteria’ for each requirement.
n Give the ‘fit criteria’ alongside the requirement. n E.g. for new ATM software
5 Requirement: “The software shall be intuitive and self-explanatory”
5 Fit Criteria: “95% of existing bank customers shall be able to withdraw money and deposit checks within two minutes of encountering the product for the first time”
n Choosing good fit criteria.
n Stakeholders are rarely this specific. n The right criteria might not be obvious:
5 Things that are easy to measure aren’t necessarily what the stakeholders want
5 Standard metrics aren’t necessary what stakeholders want
n Stakeholders need to construct their own mappings from requirements to fit
Requirements Engineering and V&V
n
Do we have elicited, understood and specified ALL
requirements?
n
Have we reached a consensus among ALL
stakeholders?
n
Are there still conflicts remaining?
n
Do we have specified ALL requirements in the
RIGHT way?
Two Levels of Requirements V&V
n At the system level
5 Ensures that the software being developed (or changed) will satisfy its stakeholders (Validation)
5 Ensures that each step followed in the process of building the software yields the right products (Verification)
n At the Requirements specification level
5 Checks the software requirements specification against stakeholders goals and requirements (Validation)
5 Checks consistency of the software requirements
specification artifacts and other software development products (design, implementation, ...) against the
When to perform Requirements V&V?
n
Need to be performed at every stage during the
(requirements) process.
5
Elicitation
g Checking back with the elicitation sources.
5
Analysis
g Checking that the domain description and requirements
are correct.
5
Specification
g Checking that the defined system requirement will
meet the user requirements under the assumptions of the domain/environment.
6 Principles to Requirements Validation
n The aim is to ensure that as much as possible errors are identified.6 principles
1. Involve with the correct stakeholders.
2. Separate errors identification from their correction.
uIndependent V&V
3. Validate requirements from different perspectives.
uReach a consensus
4. Try alternative formalisms.
uUse cases to understand goals and interactions with actors.
uState machines for transitions analysis.
uClass diagrams for architecture concerns.
Requirements V&V Techniques
n Prototyping
5 Partial implementation for validation with customers
n Inspections, reviews, walkthroughs
5 Independent inspectors search for defects and recommend
appropriate actions on agreed defects
n Simulation
5 Simulates the environment of a system and checks the
appropriateness of specified behaviors
n Formal checks
5 Completeness and sanity checks
Reviews and Inspections
n A group of people reads and analyzes requirements, look for potential problems, meet to
discuss the problems, and agree on a list of action items needed to address these problems.
A widely used requirements validation technique.
5 Lots of evidence of effectiveness of the technique
5 Standardization: IEEE 1028 Standard for Software Review and Audits
n Can be expensive
5 Careful planning and preparation
5 Pre-review checking + appropriate checklists
n Different types of reviews with varying degrees of formality
5 Reading the document: A person other than the author of the document
5 Reading and approval (sign-off): Encourages the reader to be more careful
n Walkthroughs
5 Informal, often high-level overview
5 Can be led by author/expert to educate others on his/her work
n Formal inspections
Reviews Checklists
n Essential tool for an effective review process.
5 List common problem areas and guide reviewers.
5 May include questions on several quality aspects of the document:
g comprehensibility, redundancy, completeness, ambiguity, consistency, organization, standards compliance, traceability ...
n There are general checklists and checklists for particular modeling and
specification languages.
n Checklists are supposed to be developed and maintained.
n See example on course website
5 Simple:
g http://site.uottawa.ca/~damyot/seg3101/notes/
Inspection_checklist_Wiegers.pdf
5 Complex (NASA):
Elements of a Checklist
n Comprehensibility
5 can readers of the document understand what the requirements mean?
n Redundancy
5 is information unnecessarily repeated in the requirements document?
n Completeness
5 does the checker know of any missing requirements or is there any information missing from individual requirement descriptions?
n Ambiguity
5 are the requirements expressed using terms which are clearly defined?
n Consistency
5 do the descriptions of different requirements include contradictions?
5 Are there contradictions between individual requirements and overall system requirements?
n Organisation
5 is the document structured in a sensible way?
5 Are requirements organised so that related requirements are grouped?
n Conformance to standards
References
1. Software Requirements. Karl. E. Weigers, Microsoft, 2013.
2.