VERIFICATION AND
VALIDATION – AUTOMATED
TESTING TOOLS
ABOUT ISDC
ISDC... in a nutshell:
- founded in 1990 in the Netherlands (Hilversum), 1999 in Romania
(Cluj)
- with a current pool more than 200 talented and performance-driven
people
- dedicated delivery models for end customers, IT services companies,
and IT product companies
- knowledgeable and experienced in technology stacks like
Microsoft
.NET
,
Java
,
Oracle
;
- core provided solutions:
BI
,
Application Integration
,
Application
Modernization
ABOUT ME
17/4/13
IN GOOD COMPANY 3
1. Bachelor degree - “Stefan cel Mare” University
from Suceava - Computer Science department
2. Master degree at Technical University - Cluj
Napoca – Software Engineering department
3. Working since 2011 in ISDC, 3.5 years in IT as a
Test Automation Engineer
CONTENT
1. Overview of a System Under Test
2. Why to automate things?
3. Testing levels
4. Testing types
5. Test execution (manual vs. automated path)
6. Functional testing
7. Non functional testing
8. Q & A
OVERVIEW OF A SYSTEM UNDER
TEST
17/4/13 IN GOOD COMPANY 5 SUT Component 1 Component 2 Component 3OS Hardware File System
SUT = System Under Test OS = Operating System
TESTING LEVELS IN VER/VAL
ACCEPTANCE TESTING
ACCEPTANCE TESTING
SYSTEM TESTING
SYSTEM TESTING
INTEGRATION TESTING
INTEGRATION TESTING
COMPONENT TESTING
COMPONENT TESTING
VERIFICATION
VALIDATION
TESTING TYPES - TESTING LEVELS
17/4/13 IN GOOD COMPANY 7ACCEPTANCE TESTING
ACCEPTANCE TESTING
SYSTEM TESTING
SYSTEM TESTING
INTEGRATION TESTING
INTEGRATION TESTING
COMPONENT TESTING
COMPONENT TESTING
Functional Testing
Functional Testing
Non Functional Testing
Non Functional Testing
Structural Testing
Structural Testing
FUNCTIONAL TESTING
Followed principles:
1.
We verify the compliance of the system with the functional
requirements, and we use test design techniques to assure optimal
functional coverage with our test cases
2.
Test cases are documented and managed within the test
management tools. We validate the test cases with the customer.
3.
We keep the traceability to the requirements (Requirement -> Test
Cases -> Issues)
4.
Functional test cases represent a very important input for the future
automated tests
NON FUNCTIONAL TESTING
IN GOOD COMPANY 17/4/13 9• Endurance testing
• Load testing
•
Performance testing
• Recovery testing
• Security testing
• Scalability testing
• Stress testing
• Usability testing
TEST TYPES ON ENVIRONMENTS
Environment Test Types
DEVELOPMENT Functional testing, Non functional testing
TEST Functional testing, Non functional testing
PREPRODUCTION Acceptance testing, Non functional testing
-MANUAL PATH
17/4/13
IN GOOD COMPANY 11
Requirements Design Test Cases Implement Test Cases Execute Test Cases Report Results
AUTOMATED PATH
Requirements Design / Customize Test Framework and Cases Implement Test Cases Execute Test Cases in CI Report Results CI = Continuous IntegrationTOOLS FOR FUNCTIONAL TESTING
17/4/13
IN GOOD COMPANY 13
Technology Back-end Front-end
Java Junit, TestNG, SoapUI
Selenium IDE, RC, WebDriver Java, Watij, Sahi, Test Complete
.NET MSTEST, NUnit, WCFStorm Selenium IDE, RC, WebDriver C#, Watin, Telerik, Test Complete
-AUTOMATED FRONT-END FLOW
Test Cases (Methods) Firefox Chrome Safari IE Application Generate report OperaDEMO
WORK WITH ECLIPSE, MAVEN AND
SELENIUM WEBDRIVER JAVA
1. What you need for creating and running a test
2. @Before, @Test, @After
DEMO
WORK WITH VISUAL STUDIO PREMIUM
2012, TFS AND SELENIUM WEBDRIVER
C#
1. What you need for creating and running a test
2. How the [TestClass] and [TestMethod] look like
3. Execute test method and see the results
TOOLS FOR PERFORMANCE
TESTING
17/4/13
IN GOOD COMPANY 17
Technology Tools
Java Load UI, SoapUI, JMeter
.NET Visual Studio
PERFORMANCE LEVEL
AUTOMATION
Web Server Application Server 1 Application Server 2 Database Server 1 Database Server 2 Load Agent 1 Load Agent 2 Load Agent 3 request request request request response response response responseDEMO
WORK WITH JMETER FOR MEASURING
THE PERFORMANCE
1. What you need
2. How to create a Thread and Requests to server
3. Run tests and see/collect the results
DEMO
WORK WITH JENKINS FOR
CONTINUOUS INTEGRATION
1. What you need
2. How to create a jobs
TOOLS LIST – FUNCTIONAL
TESTING
1. Junit (Java) - http://junit.org/
2. TestNG (Java) - http://testng.org
3. SoapUI (SOAP Exposed Services) - http://www.soapui.org/
4. Selenium WebDriver (many languages) - http://docs.seleniumhq.org/
5. MSTEST (.NET) - http://msdn.microsoft.com/en-us/library/ms182486.aspx
6. NUnit (.NET) - http://www.nunit.org/
7. WCFStorm (WCF Exposed Services) - http://www.wcfstorm.com
8. Telerik (.NET) - http://www.telerik.com/ or free library Web AII
TOOLS LIST – NON FUNCTIONAL
TESTING
1. SoapUI (SOAP Exposed Services) - http://www.soapui.org/
2. LoadUI - http://www.loadui.org/
3. Jmeter - http://jmeter.apache.org/
4. Microsoft Visual Studio Ultimate - http://msdn.microsoft.com/en-us/library/vstudio/dd293540.aspx
17/4/13
TOOLS LIST – TEST / ISSUE
MANAGEMENT
1. TestLink - http://www.teamst.org/
2. Jira - http://www.atlassian.com/software/jira/
3. Microsoft Test Manager - http://msdn.microsoft.com/en-us/library/vstudio/dd380763.aspx
TOOLS LIST – CONTINUOUS
INTEGRATION
1. Jenkins - http://jenkins-ci.org/
2. TFS - http://tfs.visualstudio.com/
3. Team City - http://www.jetbrains.com/teamcity/
17/4/13
CONCLUSIONS
17/4/13
THANK YOU
17/4/13
