Open Source Software
Test Tools
Norbert Jansen
Capgemini
TE3 Open Source Test Tools
Introduction
Managing consultant at Capgemini Netherlands.
Active in IT and testing for 14 years.
Experience as a tester, test manager and test consultant.
Content
Open Source Software. Test tools.
Open Source Maturity Model (OSMM).
Maturity survey of Open Source test tools.
Three examples of Open Source test tools.
Open Source Software?
Open Source has grown up
‘Linux has become a reliable alternative operating system among IT professionals. Businesses can save millions …’ (Gartner June 2004)
In 2004 Forrester surveyed 140 large companies in North America to find out their open source plans. The 60%
majority are adopting open source — and half of them use it for mission-critical applications.
What is Open Source?
Free availability of software source code.
Adjusting code is not the exclusive right of the software supplier.
Licence is about intellectual ownership and reuse. No licence fee, but not free.
Open source development method: virtual community on Internet.
Test tools
Test tools
Fault
Detection Support
Static Dynamic Management Utilities
Static analysis Test design Verification
Defect management Control of test ware Test planning Test database Test generator Performance Resource allocation Traffic monitor Coverage Dynamic analysis Link checker Test drivers Unit testing Test tools Fault Detection Support
Static Dynamic Management Utilities
Static analysis Test design Verification
Defect management Control of test ware Test planning Test database Test generator Performance Resource allocation Traffic monitor Coverage Dynamic analysis Link checker Test drivers Unit testing
Using test tools
Usage of test tools starts with selecting the proper tool.
Open Source Maturity Model (
OSMM
)Aim: to objectively value open source software.
Maturity predicts the risk that support and product development ceases to exist.
OSMM is a method for determining long list & short list in the open source software selection process.
The Open Source Maturity Model
Two pillars:
Product indicators
Measured purely on the product.
Emphasizing on process quality.
Criteria are described clearly.
Set the minimum level.
Applicability indicators
Determine applicability of a product for a user.
The Open Source Maturity Model
Product indicators
Product
age, selling points, developer community, human hierarchies, licensing
Integration
collaboration, modularity standards
Use
support, ease of deployment, Acceptance
user commodity, market penetration Applicability indicators Usability Interfacing Performance Reliability Security Proven Technology Vendor Independence Platform Independence Support Reporting Etc.
Survey of Open Source test tools
Capgemini investigated 100 test tools in Q1 2005.
Majority of tools examined were dynamic testing tools. Second largest group was test management tools.
Indicators examined:
Age, License, Human Hierarchies, Developer community, Support, Ease of Deployment, and User community.
Survey result
Maturity of Test Tools
0 2 4 6 8 10 12 14 16 18 20 0 11 13 15 17 19 21 23 25 27 N u m b e r o f to o ls
JUnit
Java Unit test tool.
Foundation for many advanced unit test frameworks such as NUnit, XMLUnit, JUnitEE, Cactus and more. Used in daily build & test at Capgemini’s Accelerated Development Centers®.
Unit and unit integration test.
Collaborates with Open Source coverage tools. Interfaces with COTS.
The Grinder
The Grinder is a pure Java load-testing framework.
Can be used for load testing any J2EE application server. The Grinder makes it easy to orchestrate the activities of a test script in many processes across many machines, using a graphical console application.
‘J2EE Performance Testing with BEA WebLogic Server’ by Peter Zadrozny, Phil Aston and Ted Osborne,
published by Expert Press.
http://sourceforge.net/projects/grinder
Bugzilla
Part of the Mozilla foundation (the Open Source community behind Firefox).
Conclusions
OS test tools certainly effective at point solutions. OS test tools frontrunners in functionality.
Apart from very mature and powerful tools, there’s a lot of garbage.
OSMM is a big help in the selection process.
Saving license cost alone not a good reason for OS test tools.
