Content
Quality, Software Quality, Quality Assurance
Software Testing
Goals of Software Testing
Defects VS Failure
Static and Dynamic Testing
Types of Software Testing
Software Testing Life Cycle
Testing and Inspection
What is Quality?
Degree to which a system, component, or process meets
1- specified requirements, and
2- customer or user needs or expectations
Other experts define quality based on:
Conformance to requirements
: the project’s
processes and products meet written specifications or
scope.
For example, if the project scope statement requires delivery of 100
computers with specific processors, memory, and so on, you could easily check whether suitable computers had been delivered
Fitness for use
: a product can be used as it was
intended
If these computers were delivered without monitors or keyboards the
What is Quality?
Customers Expectations
1.
Quality
2.
Budget
3.
Time-scale
4.
Meet the requirements
5.
Excellent performance
What is Quality?
Quality Criteria include:
Quality Assurance
Definition:
The maintenance of a desired level of quality in
a product, especially by means of attention to
every stage of the process of production.
Goal:
Quality Assurance
Quality Assurance Elements
Standards
ensure that standards are adopted and followed
Reviews and Audits
audits are reviews performed by SQA personnel to ensure hat quality guidelines are followed for all software engineering work
Testing
ensure that testing is properly planned and conducted
Error/defect Collection and Analysis
collects and analyses error and defect data to better understand how errors are introduced and can be eliminated
Changes Management
ensures that adequate change management practices have been instituted
Risk Management
Quality Assurance
SQA Tasks
1.
Prepare SQA plan for the project.
2.
Participate in the development of the project's software
process description.
3.
Review software engineering activities to verify compliance
with the defined software process.
4.
Audit designated software work products to verify
compliance with those defined as part of the software process.
5.
Ensure that any deviations in software or work products are
documented and handled according to a documented
procedure.
6.
Record any evidence of noncompliance and reports them to
07/10/20 9
Requirements
quality
Ambiguity
Completeness
Instability
Model clarity
Design quality
–
Architectural integrity
–
Component
completeness
–
Interface complexity
–
Patterns
07/10/20 10
Requirements
quality
Ambiguity
Completeness
Volatility/Instability
Model clarity
Design quality
–
Architectural integrity
–
Component
completeness
–
Interface complexity
–
Patterns
07/10/20 11
1. QA is a process oriented and focuses on defect prevention
2. QA is a set of activities for ensuring quality in the processes by which products are developed.
3. QA aims to prevent defects with a focus on the process used to make the product. It is a proactive quality process.
4. The goal of QA is to improve development and test processes so that defects do not arise when the product is being developed.
5. Verification is an example of QA. 6. QA is a managerial tool
1. Quality Control is product oriented and focuses on defect identification.
2. QC is a set of activities for ensuring quality in products. The activities focus on identifying defects in the actual products produced.
3. QC aims to identify defects in the finished product. Quality control is a reactive process.
4. The goal of QC is to identify defects after a product is developed and before it’s released.
5. Validation/Software Testing is an example of QC.
6. QC is a corrective tool.
Quality Costs
Prevention costs
quality planning, formal technical reviews,
test equipment, training
Failure costs
rework, repair, failure mode analysis
External failure costs
complaint resolution, product return and
What is Testing
The process consisting of all life cycle activities, both static and dynamic,
concerned with planning, preparation and evaluation of software products
and related work products to determine that they satisfy specified
requirements, to demonstrate that they are fit for purpose and to detect
defects.
The process of detecting defects is called testing.
Software Testing
The
process consisting of all
process
life cycle activities, both
life cycle activities
static
static
and
dynamic
dynamic
, concerned with
,
planning
planning
,
preparation
preparation
and
evaluation
evaluation of
software products and related work products
software products and related work products
to
determine that they satisfy specified requirements
determine
, to
demonstrate
demonstrate that they are fit for purpose
and to
detect
detect
defects
Static VS Dynamic Testing
Reviews, or inspections are referred to
as static testing
Actually executing
programmed code
with a given set of test cases is referred
to as
dynamic testing
Static testing involves
verification
,
whereas dynamic testing
Types of Defects that are easier to
find during static testing are:
1-
Deviation from standards
2-
Missing requirements
3-
Design defects
Dynamic Testing
The inverse of static testing is a dynamic testing, in which
code is executed.
Testing that involves the execution of the software using a
Why is it that software sometimes
do not work correctly?
Error (Mistake)
A human action that produces an incorrect result-the software is used incorrectly and
does not behave as we expected.
Defect (bug, fault)
A flaw in a component or system that can cause the component or system fail to perform
its required functions. NOT ALL DEFECTS RESULTS IN FAILURES; SOME STAY IN THE CODE AND WE NEVER NOTICE THEM.
Failure
Testing Objectives
The Major Objectives of Software Testing:
- Uncover as many as errors (or bugs) as possible in a given timeline. - Demonstrate a given software product matching its requirement specifications.
- Validate the quality of a software testing using the minimum cost and efforts.
- Generate high quality test cases, perform effective tests, and issue correct and helpful problem reports.
Major goals:
uncover the errors (defects) in the software, including errors in: - requirements from requirement analysis
- design documented in design specifications - coding (implementation)
- system resources and system environment
Verification & Validation
The distinction
between
the two terms is
largely to do with the role of specifications.
Verification
is the process of checking that
the software meets the specification.
Verification & Validation
Verification
1.Have you built the product right?
2.Does the product meet system specification?
Validation
1.Have you built the right product?
Testing Principle-Testing is Context
Dependent
It is done differently in different contexts (nature of the
software)
1.
Safety software is tested differently from an e-commerce site.
2.
University software are tested differently from banking
software.
EXAMPLE 1
How many tests would you need to do to
completely test a one-digit numeric field?
1.
10 possible valid numeric values (10 tests)
2.
We need to ensure that all invalid values are rejected.
e.g. 26 uppercase letters, 26 lowercase letters,
When we can meet our test
objectives?
Testing Principle-Early Testing
Testing activities should start as early as possible in the
Focusing on defects can help us
plan our tests
Testing Principle- Defect Clustering
A small number of modules contain most of the
defects discovered during testing.
Reasons of defect clustering
1-Code is complex
2-Code is tricky
3-Not reviewing properly during development
Debugging Removes Defects
Testing Principle-Pesticide Paradox
If the same tests are repeated over and over again,
Is the software defects free?
Testing Principle- Testing shows presence of
defects
Testing can show that defects are present, but can
If we don’t find defects does that mean
the user will accept the software?
Testing Principle- Absence of errors fallacy
Finding and fixing defects does not help if the system built
Testing Principles-SUMMARY
Testing shows presence of defects
2- Exhaustive testing is impossible
3- Early testing
4- Defect clustering
5- Pesticide Paradox
6- Testing is context dependent
Test Levels-Component Testing
Component Testing
“The testing of individual software
components is called component testing.”
It searches for defects and verifies the functioning
Test Levels-Component Testing
Component Testing is classified into 3 types:
Execution Testing
Verifying the logic of the program.(statement coverage,
loop coverage, conditional coverage, program technique
coverages etc.)
Operation Testing
Checking whether the program is running on the customer expected
platform(i.e. OS, Compilers, Browser etc.)
Mutation Testing
Test Levels-Component Testing
Approach of Component Testing
Extreme Programming (XP)
Test-First Approach OR Test-Driven Development
To prepare test cases before coding.
OR
A way of developing software where the test cases
Test Levels-Component Testing
Component testing may include testing of:
1- Functionality Requirements
2- Non-functionality Requirements
3- Robustness Testing
Robustness
to which a component or system can function correctly in the
presence of invalid inputs.
Robustness testing
Test Levels-Integrated Testing
Integration
The process of combining components into larger
assemblies.
Integrated Testing
“All individual units are integrated/combined and
Test Levels-Integrated Testing
Other Levels of Integration Testing
Component Integration Testing
Tests the interactions between software components and is
done after component testing.
System Integration Testing
Tests the interactions between different systems i.e. systems
that can even run on different platforms.
Big-bang Integration Testing (First Extreme)
All components are integrated simultaneously, after which
Advantages of Big-bang Integration Testing
No need to simulate parts, often called stubs.
Stubs
A stub is a piece of software which performs functionality of
a not yet constructed/under- constructed component.
OR
while integrating in top down approach any compulsory
Disadvantages of Big-bang Integration
Testing
1-
It is time consuming.
2-
Difficult to trace the cause of failures
because of late integration.
3-
Difficult to check the functionality of
Second Extreme
All components are integrated one by one and tests
Test Levels-System Testing
“The process of testing an integrated system
to
verify
that
it
meets
specified
requirements.”
1.
It concerned with the behavior of the whole
system.
2.
It is most often the final test to verify that
the system to be delivered meets the
specification.
3.
Its purpose is to find as many defects as
System testing should investigate:
Functional Requirements
A requirement that a component or system must perform i.e
interoperability, security, accuracy and compliance.
Non-Functional Requirements
Requirement that does not relate the functionality, but to
attributes of such as:
Reliability
Efficiency
Usability
Test Levels-Acceptance Testing
“Its an activity of testing whether the software is
Other types of acceptance testing
Compliance Acceptance Testing OR Regulation Acceptance Testing
The capability of the software product to strictly firm to Governmental
standards, regulations in laws and safety.
Commercial-Off-The-Shelf (COTS) software acceptance testing
1-
Alpha Testing:
This takes place at the developer’s site. A cross-section of users and
member’s of the developer’s organization are invited to use the system.
2-
Beta Testing
