AppLabs Technologies AppLabs Technologies 1700 Market Street, Suite 1406
1700 Market Street, Suite 1406 Philadelphia, PA 19103Philadelphia, PA 19103 Phone:
TABLE OF CONTENTS TABLE OF CONTENTS 1.0 1.0 Approach.Approach... 33 1.1 1.1 Overview...Overview... 33 1.2
1.2 Definition Definition of of ScopeScope ... 33 1.3
1.3 Tests Tests to to be be ConductedConducted ... ... 33 1.4
1.4 Tools Tools Utilized...Utilized... ... 55 1.5
1.5 Project Project Inception Inception ChecklistChecklist ... ... 55 2.0
2.0 Test Test PlansPlans ... 66 2.1 Tes
2.1 Test t Plan Plan Creation Creation Process...Process... ... 66 2.2
2.2 Test Test Plan Plan StructureStructure... 77 2.3
2.3 Sample Sample Test Test Plan...Plan... ... 99
3.0 Test Plan Execution
3.0 Test Plan Execution... ... 1010 3.1
3.1 Manual Manual Execution Execution Process...Process... ... 1111 3.2
3.2 Reporting Reporting DetailsDetails ... 1212 3.3
3.3 Handling Handling ProceduresProcedures... ... 1313 3.4
3.4 Severity Severity Levels...Levels... ... 1414 4.0
4.0 Fix Fix Validation Validation ProcessProcess ... ... 1515 5.0
5.0 Reporting...Reporting... ... 1717 5.1
5.1 Daily Daily Summary Summary Data...Data... 1717 5.2
5.2 Weekly Weekly Summary Summary Report...Report... ... ... 1818 5.3
5.3 End End of of Cycle Cycle ReportsReports ... ... 1818 6.0
6.0 Compatibility Compatibility TestTest ... ... ... 1919 6.1
6.1 Browser Browser ChecksChecks ... ... 1919 6.2
6.2 Operating Operating SystemsSystems ... 1919 7.0
7.0 Automated Automated Testing...Testing... .... 1919 8.0 S
8.0 Stress, tress, Load and Load and Capacity Capacity Testing...Testing... 1919 8.1
8.1 Load Load TestingTesting ... 2020 8.2
8.2 Stress Stress TestingTesting ... ... 2020 8.3
8.3 Capacity Capacity TestingTesting ... ... ... 2020 8.4
8.4 Reporting Reporting TermsTerms ... ... 2020 9.0
9.0 Minimal Minimal Acceptance Acceptance Tests...Tests... ... 2121 10.0
1.0 Approach
1.0 Approach
1.1 Overview 1.1 Overview
AppLabs Technologies provides offshore and
AppLabs Technologies provides offshore and on-site quality assurance services. on-site quality assurance services. We have We have aa dedicated staff of QA professionals to perform test plan creation, script development for automated dedicated staff of QA professionals to perform test plan creation, script development for automated testing, manual and automated test plan execution, and stress, load and capacity testing.
testing, manual and automated test plan execution, and stress, load and capacity testing. The following pages outline our
The following pages outline our overall, fundamental approach to QA. overall, fundamental approach to QA. While each projectWhile each project varies we recommend, and utilize the following procedures outlined in this document. varies we recommend, and utilize the following procedures outlined in this document. 1.2 Definition of Scope
1.2 Definition of Scope
A definition of scope document is created to describe the QA phase and schedule for each release. A definition of scope document is created to describe the QA phase and schedule for each release. The goal of the scope document is to achieve agreement and commitment from all departments The goal of the scope document is to achieve agreement and commitment from all departments involved to the
involved to the outlined conditions and outlined conditions and timelines identified. timelines identified. The scope The scope document contains:document contains: Areas to be tested
Areas to be tested •
• New System ModulesNew System Modules •
• Modified System ModulesModified System Modules •
• Unchanged System ModulesUnchanged System Modules Areas not to be tested
Areas not to be tested •
• System Module – ReasonSystem Module – Reason
(i.e. Customer Service – Internal application, Unchanged) (i.e. Customer Service – Internal application, Unchanged) Environment Requirements
Environment Requirements •
• Connection thru: [VPN, Firewall Access, DNS]Connection thru: [VPN, Firewall Access, DNS] •
• DB Access Required & Provided [Y/N]DB Access Required & Provided [Y/N] •
• Platforms Needed [Win 95, Win 98, Win NT, Win ME, Win 2000,Platforms Needed [Win 95, Win 98, Win NT, Win ME, Win 2000, Solaris, Mac 8.6, Mac 9, Mac 10]
Solaris, Mac 8.6, Mac 9, Mac 10] •
• Browser Types Needed [IE 4.0, IE 4.1, IE 5, IE 5.5, NS 4.08, NS 4.75, NSBrowser Types Needed [IE 4.0, IE 4.1, IE 5, IE 5.5, NS 4.08, NS 4.75, NS 6.0]
6.0] 1.3 Tests to be Conducted 1.3 Tests to be Conducted
After identifying the areas of the application to be
After identifying the areas of the application to be tested, the type of tested, the type of tests must be defined. tests must be defined. TheThe below type of tests related to Black Box testing are considered.
below type of tests related to Black Box testing are considered. Functional Tests
Functional Tests
A set of functional tests will developed based on client documentation, and review of any A set of functional tests will developed based on client documentation, and review of any existing systems.
existing systems. Dedicated QA Dedicated QA Engineers will execute these Engineers will execute these functional tests manually.functional tests manually. Compatibility Tests
Compatibility Tests
AppLab’s QA lab provides for testing to be conducted on multiple browsers and AppLab’s QA lab provides for testing to be conducted on multiple browsers and operating systems.
operating systems. The desired The desired testing combinations and testing combinations and process (including priority of process (including priority of testing) will be determined during the scope definition process.
testing) will be determined during the scope definition process. Stress & Performance Tests
Stress & Performance Tests
We use market-leading tools to assess how the site/product holds up under different We use market-leading tools to assess how the site/product holds up under different loads and we provide the customer with extensive data to help scale the site/product up loads and we provide the customer with extensive data to help scale the site/product up to meet higher loads.
Automated Testing Automated Testing
AppLabs has the capacity to devote resource(s) dedicated to developing automated test AppLabs has the capacity to devote resource(s) dedicated to developing automated test scripts. The areas of the application where automated testing are applied will be
scripts. The areas of the application where automated testing are applied will be identified by AppLabs and
identified by AppLabs and the client during the the client during the scoping phase. scoping phase. Typically, areas whichTypically, areas which remain relatively constant and that are considered reasonably stable are considered remain relatively constant and that are considered reasonably stable are considered strong candidates for automated testing.
1.4 Tools Utilized 1.4 Tools Utilized
Bug Tracking Applications Bug Tracking Applications
Bug tracking applications are employed to provide a common database of issues reported. Bug tracking applications are employed to provide a common database of issues reported. The bug tracking application will be used to by QA Engineers to report bugs, assign severity The bug tracking application will be used to by QA Engineers to report bugs, assign severity levels and to
levels and to close bugs. close bugs. Our clients will utilize the bug Our clients will utilize the bug tracking application to report itemstracking application to report items as fixed or to defer bugs (to be fixed in later releases).
as fixed or to defer bugs (to be fixed in later releases).
For each QA project, AppLabs will provide access to our recommended bug tracking For each QA project, AppLabs will provide access to our recommended bug tracking application
application, Bugzilla. , Bugzilla. If our client is already utilizing a bug tracking application, AppIf our client is already utilizing a bug tracking application, AppLabs willLabs will instead enter bugs
instead enter bugs directly into that system. directly into that system. AppLabs has considerable experience with theAppLabs has considerable experience with the following bug tracking applications:
following bug tracking applications: •
• Teamshare’s TeamtrackTeamshare’s Teamtrack •
• Soffront’s TrackWebSoffront’s TrackWeb •
• Softwise’s PR-Tracker Softwise’s PR-Tracker •
• GNATs Bug TrackingGNATs Bug Tracking Automation Testing Tools Automation Testing Tools
AppLabs utilizes Mercury Interactive’s WinRunner to perform automated testing. The AppLabs utilizes Mercury Interactive’s WinRunner to perform automated testing. The
decision to apply automated testing tools and the areas of the application on which to apply decision to apply automated testing tools and the areas of the application on which to apply them will be defined during the scope definition phase of the project.
them will be defined during the scope definition phase of the project. Stress, Load and Capacity Testing Tools
Stress, Load and Capacity Testing Tools
To perform stress, load and capacity testing, AppLabs applies WinRunner Scripts to To perform stress, load and capacity testing, AppLabs applies WinRunner Scripts to Mercury Interactive’s LoadRunner.
Mercury Interactive’s LoadRunner. Using LoadRunner, Using LoadRunner, AppLabs can AppLabs can simulate activitiessimulate activities being performed by
being performed by varying numbers of users. varying numbers of users. AppLabs provides our clients reports AppLabs provides our clients reports onon system bottlenecks, break points and performance measures.
system bottlenecks, break points and performance measures. 1.5 Project Inception Checklist
1.5 Project Inception Checklist
AppLabs will manage the below checklist, which tracks the information transfer concerning product AppLabs will manage the below checklist, which tracks the information transfer concerning product documentation, testing documentation and tools, application access and application knowledge documentation, testing documentation and tools, application access and application knowledge transfer. transfer. Client: Client: Project Name: Project Name: Main
Main Area Area Item Item Onus Onus Date Date InfoInfo
Web
Web Based Based System System Available Available to to All All Parties Parties AppLabsAppLabs Client
Client Users Users Created Created AppLabsAppLabs Bug Tracking
Bug Tracking
AppLabs
AppLabs Users Users Created Created AppLabsAppLabs Functional
Functional Requirements Requirements Document(s) Document(s) to to QA QA ClientClient Users
Users Manual Manual to to QA QA (if (if available) available) ClientClient Documentation
Documentation
Old
Old Test Test Plans Plans to to QA QA (if (if available) available) ClientClient URL
URL for for QA QA EnvironmenEnvironment t ClientClient URL
URL for for Staging Staging Environment Environment ClientClient System
System User User Names/PasswNames/Passwords ords ClientClient Testing
Testing Environment Environment
QA Lab Resources Allocated (OS/Browser etc)
QA Lab Resources Allocated (OS/Browser etc) AppLabsAppLabs Product
Product Walkthrough Walkthrough ClientClient Bug
Bug Handling Handling Procedures Procedures AppLabs AppLabs // Client Client Communication
Communication
Emergency
Emergency Contact Contact Information Information AppLabs AppLabs // Client Client
General Contact Information (recipients of daily status General Contact Information (recipients of daily status reports)
reports)
Client Client Schedule
Schedule Scheduled Scheduled Delivery Delivery of of First First Build Build ClientClient
2.0 Test Plans
2.0 Test Plans
2.1 Test Plan Creation Process 2.1 Test Plan Creation Process
AppLabs takes considerable care in developing and managing test plans and test cases. AppLabs takes considerable care in developing and managing test plans and test cases.
Developing accurate, clear and concise test cases is critical to the success of black box, functional Developing accurate, clear and concise test cases is critical to the success of black box, functional testing.
testing.
Client Documentation Client Documentation
The first step in developing test cases is receiving and reviewing the Functional The first step in developing test cases is receiving and reviewing the Functional Requirements Documents provided by the
Requirements Documents provided by the client. client. The FRD review process includes: of The FRD review process includes: of FRDs:
FRDs: •
• Unit Level Unit Level FunctionalitFunctionality dissectedy dissected •
• Understanding of each pages’ rendering logicUnderstanding of each pages’ rendering logic •
• Creation of Application flowCreation of Application flow •
• Integration ScenariosIntegration Scenarios
Test Plan Sign-off Process Test Plan Sign-off Process
AppLabs follows the below diagram in order to gain client approval of the test cases AppLabs follows the below diagram in order to gain client approval of the test cases developed by AppLabs.
developed by AppLabs. The basic tenets The basic tenets of the of the sign-off process include:sign-off process include: •
• Developing Test Plan Indexes:Developing Test Plan Indexes: •
• Developing Detailed Test Scenarios (test cases):Developing Detailed Test Scenarios (test cases):
•
• Review and Adjustment cycle with client:Review and Adjustment cycle with client: •
Receipt of Client Receipt of Client Documentation Documentation
Test Case Index Test Case Index Prepared & Prepared & Sent to Client Sent to Client Detailed Test Detailed Test Cases Prepared & Cases Prepared & Sent To Client Sent To Client Client Client Approves? Approves?
Test Case Index Test Case Index Re-Written / Re-Written / Changes Made & Changes Made & Sent to Client Sent to Client Client Client Approves? Approves? Test Plan Test Plan Re-Written / Re-Written / Changes Made & Changes Made & Sent to Client Sent to Client Client Client Approves? Approves? No No Yes Yes No No Yes Yes Client Client Approves? Approves? No No
Test Plan Sign Off Test Plan Sign Off Received From Received From Client Client Yes Yes Yes Yes No No
First Bug Report First Bug Report & Completed & Completed Test Plans Test Plans Delivered to Delivered to Client Client Formal Testing Formal Testing Begins Begins
Test Plan Version Integrity Test Plan Version Integrity
In order to maintain the integrity of the testing, AppLabs places the accepted version placed In order to maintain the integrity of the testing, AppLabs places the accepted version placed as read only
as read only in specified file location. in specified file location. Any modifications/additions to test plan duringAny modifications/additions to test plan during execution are sent to client and maintained in ‘working’ version of test plan.
execution are sent to client and maintained in ‘working’ version of test plan.
2.2 Test Plan Structure 2.2 Test Plan Structure
AppLabs follows a strict structure for
AppLabs follows a strict structure for our test plans. our test plans. The test plan The test plan structure is designed to structure is designed to provideprovide quick reporting and to assist in problem tracking.
Front Page Front Page
Each test plan contains a first page, which contains the following for each section and for Each test plan contains a first page, which contains the following for each section and for the entire test plan:
the entire test plan: # Test Cases, # # Test Cases, # Cases Completed, # Passed, # Cases Completed, # Passed, # Failed, # Blocked, %Failed, # Blocked, % Completed, % Passed, % Failed, % Blocked.
Completed, % Passed, % Failed, % Blocked.
TOTAL
TOTAL TESTS TESTS TOTAL TOTAL PERCENTPERCENT
#
# Tests: Tests: 540540
#
# Completed: Completed: 242 242 % Complete:% Complete: 45%45%
#
# Passed: Passed: 155 155 % % Passed:Passed: 29%29% #
# Failed: Failed: 87 87 % % Failed:Failed: 16%16%
#
# To To be be executed executed 298 298 % % To To be be executedexecuted 55%55% #
# Blocked: Blocked: 12 12 % % Blocked:Blocked: 2%2%
Column Headings Column Headings
Each detailed test scenario (test
Each detailed test scenario (test case) has the case) has the following column headings: following column headings: Test Case ID,Test Case ID, Status, Defect #, Test Case Area, Test Steps, Expected Results
Status, Defect #, Test Case Area, Test Steps, Expected Results
Test Case Test Case
id
id Status Status Defect Defect # # Test Test Area Area Reference Reference Id Id Test Test Steps Steps Expected Expected ResultResult
Test Areas Test Areas
All AppLab’s test plans a
All AppLab’s test plans are divided into sections. re divided into sections. Sections consist of groupings of Sections consist of groupings of testtest scenarios.
scenarios. Examples of Examples of individual test plan sections individual test plan sections include: Registration, Login, User include: Registration, Login, User Rights & Privileges,
Rights & Privileges,
Se
Sectctioion n #T#Tesests ts % % CoCompmpleletete % % PaPasssseded %F%Faiaileled d %B%Blolockckeded
1.
1. Login Login 11 11 100% 100% 100% 100% 0% 0% 0%0%
2.
2. Deployment Deployment Set Set 20 20 100% 100% 95% 95% 5% 5% 0%0%
3.
3. Views Views 52 52 98% 98% 98% 98% 2% 2% 2%2%
4.
4. Edit Edit from from Package Package view view 19 19 100% 100% 100% 100% 0% 0% 0%0% 5.
5. Edit Edit from from File File view view 15 15 100% 100% 67% 67% 33% 33% 0%0% 6.
6. Edit Edit from from Host Host view view 13 13 100% 100% 100% 100% 0% 0% 0%0% 7.
7. File File Menu Menu Options Options 23 23 96% 96% 0% 0% 0% 0% 4%4%
8.
8. Other Other Options Options 4 4 100% 100% 100% 100% 0% 0% 0%0%
9.
9. Package Package Settings Settings 19 19 100% 100% 89% 89% 11% 11% 0%0%
10.
10. File File Settings Settings 28 28 100% 100% 89% 89% 11% 11% 0%0%
11.
11. Host Host Settings Settings 27 27 100% 100% 100% 100% 0% 0% 0%0%
12.
12. Log Log 20 20 100% 100% 100% 100% 0% 0% 0%0%
13.Deploy
13.Deploy 35 35 100% 100% 100% 100% 0% 0% 0%0%
14.
14. Server Server Configurations Configurations 57 57 100% 100% 49% 49% 51% 51% 0%0% 1
155. . AAddmmiinniissttrraattiioon n CCoonnffiigguurraattiioonnss 881 1 8855% % 5588% % 4422% % 1155%% 16.
16. Scripts Scripts 15 15 100% 100% 67% 67% 33% 33% 0%0%
17.
17. Help Help 77 77 100% 100% 91% 91% 9% 9% 0%0%
18.
2.3 Sample Test Plan 2.3 Sample Test Plan
AppLabs test plans are divided into two components, the test plan index and the collection of AppLabs test plans are divided into two components, the test plan index and the collection of detailed test scenarios.
detailed test scenarios. Test Plan Index Test Plan Index
The test plan index is developed to provide a high level overview of the application sections The test plan index is developed to provide a high level overview of the application sections and the type of tests to be performed.
and the type of tests to be performed. The test plan index acts as a table of contents for theThe test plan index acts as a table of contents for the final collection of test
final collection of test scenarios.scenarios.
A sample of one section of a test plan index is provided. A sample of one section of a test plan index is provided.
Detailed Test Cases Detailed Test Cases
Our detailed test scenarios are developed so each QA engineer will follow the exact Our detailed test scenarios are developed so each QA engineer will follow the exact pre-defined steps in order
defined steps in order to isolate and to isolate and test specific components of the test specific components of the application. application. As testingAs testing is conducted, new test scenarios may be
is conducted, new test scenarios may be added. added. These cases are added in These cases are added in their their appropriate section and once developed are completed for each testing cycle. appropriate section and once developed are completed for each testing cycle. One detailed test scenario is shown below.
One detailed test scenario is shown below.
5 Edit Deployment set option for
5 Edit Deployment set option for objectobject 5.1 Add File to deployment set
5.1 Add File to deployment set 5.1.1 Add One
5.1.1 Add One Available File Available File (With host and (With host and Package)Package) 5.1.2 Add One
5.1.2 Add One Available File Available File (with package only)(with package only) 5.1.3 Add All Available File
5.1.3 Add All Available File 5.1.4 Create New file and Add 5.1.4 Create New file and Add 5.2 Delete File from deployment set 5.2 Delete File from deployment set
5.2.1 Delete Existing File 5.2.1 Delete Existing File 5.2.2 Delete Newly Added File 5.2.2 Delete Newly Added File 5.3 Add Package to deployment set 5.3 Add Package to deployment set
5.3.1 Add One Available Package 5.3.1 Add One Available Package 5.3.2 Add All Available packages 5.3.2 Add All Available packages 5.4 Delete Package from deployment set 5.4 Delete Package from deployment set
5.4.1 Delete Existing Package 5.4.1 Delete Existing Package 5.4.2 Delete Newly Added Package 5.4.2 Delete Newly Added Package 5.5 Add Host to deployment set
5.5 Add Host to deployment set 5.5.1 Add One Available Host 5.5.1 Add One Available Host 5.5.2 Add All Available Hosts 5.5.2 Add All Available Hosts 5.5.3 Create New Host and Add 5.5.3 Create New Host and Add 5.6 Delete Host from deployment set 5.6 Delete Host from deployment set
5.6.1 Delete Existing Host 5.6.1 Delete Existing Host 5.6.2 Delete Newly Added Host 5.6.2 Delete Newly Added Host
Test Case Test Case
id
id Status Status Defect Defect ##
Test Area Test Area Reference Id
Reference Id Test Test Steps Steps Expected Expected ResultResult 5.1.1
5.1.1 File - Add - withFile - Add - with host and with host and with Package Package
1. Launch the Kelly application 1. Launch the Kelly application 2. Login with the Valid User 2. Login with the Valid User 3. Select a Deployment Set Icon to
3. Select a Deployment Set Icon to be openedbe opened 4. From the Main Menu, Select the Files View Tab 4. From the Main Menu, Select the Files View Tab 5. Select the Deployment Set Radio button 5. Select the Deployment Set Radio button 6. Click the Add Link
6. Click the Add Link
7. In the Select Objects Screen, Select a
7. In the Select Objects Screen, Select a File (having PackagesFile (having Packages and Hosts)
and Hosts)
in the Available List Box in the Available List Box 8. Click > Button, then Click
8. Click > Button, then Click the Submit Files Buttonthe Submit Files Button
9. Verify this (Result: Selected File should be listed under the 9. Verify this (Result: Selected File should be listed under the Deployment Set in the Main Menu Screen of the File view) Deployment Set in the Main Menu Screen of the File view) 10. Click on the Package View tab
10. Click on the Package View tab
11. Verify this (Result: Selected File should be
11. Verify this (Result: Selected File should be listed below thelisted below the Packages under the Deployment Set in the Main Menu Screen Packages under the Deployment Set in the Main Menu Screen of the Files view)
of the Files view)
12. Click on the Hosts View tab 12. Click on the Hosts View tab
13. Verify this (Result: Selected File should be
13. Verify this (Result: Selected File should be listed below thelisted below the Hosts under the Deployment Set in the Main Menu Screen of Hosts under the Deployment Set in the Main Menu Screen of the Hosts view)
the Hosts view)
14. In the Main Menu Screen, click the log link 14. In the Main Menu Screen, click the log link
15. Select Date of Creation from Date Dropdown; username from 15. Select Date of Creation from Date Dropdown; username from the user Dropdown
the user Dropdown 16. Click the Apply Filter 16. Click the Apply Filter buttonbutton
17. Verify this (Result: Information for the adding
17. Verify this (Result: Information for the adding of this Fileof this File should be displayed)
should be displayed)
18. Click on Return to main Screen 18. Click on Return to main Screen
19. In the Main Menu Screen, Click on deploy link 19. In the Main Menu Screen, Click on deploy link 20. select the option "full"
20. select the option "full"
21. click Proceed to Deployment button 21. click Proceed to Deployment button
22. Verify this (Result: Deployment request Sent Screen should be 22. Verify this (Result: Deployment request Sent Screen should be displayed
displayed
and then the Log screen should be displayed) and then the Log screen should be displayed) 23. From the Log
23. From the Log Screen, Select the Date of deployment from DateScreen, Select the Date of deployment from Date dropdown;
dropdown;
Username from the User
Username from the User Dropdown; Deployment fromDropdown; Deployment from the Action Dropdown
the Action Dropdown
24. Verify this (Result: Information Regarding this Deployment 24. Verify this (Result: Information Regarding this Deployment should be displayed)
should be displayed)
1. Check the File view (Step 9) 1. Check the File view (Step 9) 2. Check the Package View (Step 11) 2. Check the Package View (Step 11) 3. Check the Host View (Step 13) 3. Check the Host View (Step 13) 4. Check the Log file for the added File 4. Check the Log file for the added File (Step 17)
(Step 17) 5. Check the
5. Check the Log file for theLog file for the deploymentdeployment status (step 24)
status (step 24)
3.0 Test Plan Execution 3.0 Test Plan Execution AppLabs QA
AppLabs QA engineers follow a strict process, while performing manual engineers follow a strict process, while performing manual testing. testing. The execution of The execution of test cases involves the following steps.
3.1 Manual Execution Process 3.1 Manual Execution Process
Applabs QA Engineers follow the process shown below in executing test cases and in compiling Applabs QA Engineers follow the process shown below in executing test cases and in compiling end of
end of day summary data. day summary data. Details regarding each step are Details regarding each step are highlighted below.highlighted below.
Test Case Test Case Executed Executed Result Result "P" entered in "P" entered in Status Column of Status Column of
Test Plan Test Plan Fail Fail "F" entered in "F" entered in Status Column of Status Column of
Test Plan Test Plan
Bug Entered into Bug Entered into
BTS BTS Bug ID Number Bug ID Number from BTS Entered from BTS Entered
into Test Plan into Test Plan Against Failed Against Failed Test Case Test Case Section Summary Section Summary Reviewed for Reviewed for Accuracy Accuracy
Process (A) Repeated Process (A) Repeated
Until Section Until Section is Completed is Completed A A Section Section Summaries Summaries Combined Combined
Test Plan Main Test Plan Main
Summary Summary Reviewed for Reviewed for Accuracy Accuracy "B" entered in "B" entered in Status Column of Status Column of
Test Plan Test Plan Reason for Reason for Blocked? Blocked? Tester Prepares Tester Prepares Question along Question along with Test Case with Test Case
Number Number Bug Number Bug Number Entered in Defect Entered in Defect Column of Test Column of Test Plan Plan Question Sent to Question Sent to Client in Client inDailyDaily Status Report Status Report Blocked by Bug Blocked by Bug Blocked Report Blocked Report Sent in Sent inDailyDaily Status Report Status Report Test Plan Test Plan Summary Data Summary Data Sent in Sent inDailyDaily Status Report Status Report Executed Test Executed Test Plans Sent As Plans Sent As Attachment w/ Attachment w/ Daily Status Daily Status Report Report P Paassss BBlloocckkeedd Info Info Needed Needed B
•
• Execute scenarios from top to bottom on assigned browser/platformExecute scenarios from top to bottom on assigned browser/platform •
• Enter Pass, Fail, Blocked or Question in Results ColumnEnter Pass, Fail, Blocked or Question in Results Column
o
o If Fail, open bug tracker If Fail, open bug tracker
!
! Enter steps performed into bug tracker & write descriptionEnter steps performed into bug tracker & write description !
! Repeat steps to verify bug occurs before submittingRepeat steps to verify bug occurs before submitting !
! Perform steps on other platform/browser combos per pre-Perform steps on other platform/browser combos per
pre-arrangement with client arrangement with client
!
! Add Additional Notes to Bug Tracker, SubmitAdd Additional Notes to Bug Tracker, Submit !
! Write bug number in “Default ID” columnWrite bug number in “Default ID” column !
! Upon completion of each section, fill in section detailsUpon completion of each section, fill in section details
o
o If BlockedIf Blocked
!
! Enter the default number of issue which is blocking test caseEnter the default number of issue which is blocking test case !
! If no specific issue exists, open issue, record number and enter inIf no specific issue exists, open issue, record number and enter in
result for blocked issue result for blocked issue
o
o If QuestionIf Question
!
! Enter a detailed doubt in the result columnEnter a detailed doubt in the result column !
! Before the end of Before the end of each day each day , consult AppLabs QA manager , consult AppLabs QA manager !
! If the issue cannot be resolved and scenario tested, a detailedIf the issue cannot be resolved and scenario tested, a detailed Clarification Request
Clarification Request will be sent to client concerning that issue.will be sent to client concerning that issue. •
• Each days test plans will be saved in a dated folder Each days test plans will be saved in a dated folder •
• All bug numbers will be carried over from previous days testing and placed in theAll bug numbers will be carried over from previous days testing and placed in the default number so that multiple bugs are not opened on the same issue
default number so that multiple bugs are not opened on the same issue •
• When bugs are closed, the closed date will be added underneath the default number.When bugs are closed, the closed date will be added underneath the default number. •
• Before execution of a Test Plan Before execution of a Test Plan begins. begins. The previous day's test plan is modified allThe previous day's test plan is modified all data in
data in "Status" column "Status" column are removed. are removed. ALL data ALL data in "Defect" in "Defect" column remains. column remains. UponUpon completion of 1 Full Cycle (all cases executed or blocked), the daily executed plans completion of 1 Full Cycle (all cases executed or blocked), the daily executed plans are combined for an 'End of Cycle" report.
are combined for an 'End of Cycle" report. 3.2 Reporting Details
3.2 Reporting Details AppLabs placing a
AppLabs placing a strong emphasis on strong emphasis on reporting identified issues correctly and thoroughly. reporting identified issues correctly and thoroughly. In order In order to provide our clients with the best possible information concerning an issue, we take the following to provide our clients with the best possible information concerning an issue, we take the following steps while reporting quality assurance issues.
steps while reporting quality assurance issues. •
• Identify: Following Test cases & logical thinkingIdentify: Following Test cases & logical thinking
o
o Test cases provide aTest cases provide a basisbasisfor testingfor testing o
o Other scenarios should be run as application grows; these should be addedOther scenarios should be run as application grows; these should be added
to the test cases to the test cases •
• Investigate: After the initial bug is foundInvestigate: After the initial bug is found
o
o Investigate opportunities to eliminate variablesInvestigate opportunities to eliminate variables o
o Identify associated pages or areas where the immediate bug shows impactIdentify associated pages or areas where the immediate bug shows impact o
o Perform ‘gray’ box investigating by accessing the database (if applicable)Perform ‘gray’ box investigating by accessing the database (if applicable) o
o Rework the bug until its impact if fully understood.Rework the bug until its impact if fully understood.
•
• Reporting: Reporting: AppLabs QA AppLabs QA engineers follow engineers follow a a strict reporting process. strict reporting process. This includesThis includes the reporting the following for each item.
the reporting the following for each item.
o
o Date ReportedDate Reported o
o Reported ByReported By o
o Version Reported IN (build number)Version Reported IN (build number) o
o Environment (which section of code stream - QA, INT, DEV, Production)Environment (which section of code stream - QA, INT, DEV, Production) o
o TitleTitle o
o
o Browser(s)Browser(s) o
o Type (bug, enhancement)Type (bug, enhancement) o
o Severity (as reported by QA engineer)Severity (as reported by QA engineer) o
o Detailed Description – Elements (with date/personnel stamp)Detailed Description – Elements (with date/personnel stamp)
!
! Login: provide user name, password, access rights, group etc..Login: provide user name, password, access rights, group etc.. !
! Landing Page: Define page you are taken to.Landing Page: Define page you are taken to. !
! Navigation: Define link/selection area & link/selection chosenNavigation: Define link/selection area & link/selection chosen !
! Navigation: Define landing page, describe page (continue 3 & 4 asNavigation: Define landing page, describe page (continue 3 & 4 as
required) required)
!
! Issue: Identify the Issue: Identify the outcome of outcome of the sequencethe sequence !
! Issue Clarification: Identify the expected resultsIssue Clarification: Identify the expected results !
! Issue Exposure: Identify how this impacts every area - other screens,Issue Exposure: Identify how this impacts every area - other screens,
database etc. database etc.
!
! Issue Wrap-up: Specific Details eliminating variables or providingIssue Wrap-up: Specific Details eliminating variables or providing
developers insight into problem developers insight into problem
!
! Testing Wrap-up: when this was last tested, when it last worked, how itTesting Wrap-up: when this was last tested, when it last worked, how it
last worked etc. last worked etc.
!
! Final Emphasis: Specific Environments, Specific User Groups, anythingFinal Emphasis: Specific Environments, Specific User Groups, anything
previously stated which needs more emphasis previously stated which needs more emphasis
!
! Screen shots where/when applicableScreen shots where/when applicable
3.3 Handling Procedures 3.3 Handling Procedures
In order to accurately identify, investigate and report problem issues over the course of multiple In order to accurately identify, investigate and report problem issues over the course of multiple testing cycles, AppLabs executes a thorough process once a problem issues has been identified. testing cycles, AppLabs executes a thorough process once a problem issues has been identified. This process is shown in the below diagram.
This process is shown in the below diagram. The below flow is used
The below flow is used when a problem issue is when a problem issue is found, while executing the test plans. found, while executing the test plans. The diagramThe diagram is not a flow
is not a flow designed to review fixed bugs. designed to review fixed bugs. That flow is outlined in the "Fix Verification” section of That flow is outlined in the "Fix Verification” section of this document.
Problem Issue Problem Issue Identified Identified Bug ID in Bug ID in Defect Defect Column? Column? Bug Details in Bug Details in Bug Tracker Bug Tracker Reviewed by Reviewed by Tester Tester Add'l Info Add'l Info Required? Required? Open? Open? Check Current Check Current Bug Status in Bug Status in BTS BTS
Bug Annotated & Bug Annotated & Report, Sent w/ Report, Sent w/ Daily Status Daily Status Report Report "F" entered in "F" entered in Status Column Status Column Bug Bug Re-Opened Opened Date, Build # Date, Build # & Comments & Comments in Resolution in Resolution Field Field Bug Bug Re-Opened (or) Opened (or) New Bug New Bug Entered Entered
This will be based on Discussion w/ This will be based on Discussion w/ Client.
Client. Will depend on Will depend on whether Bug waswhether Bug was closed during "this" testing cycle AND closed during "this" testing cycle AND
process to close specific bug. process to close specific bug. Re-Opened Re-Opened List Sent to List Sent to Client in Client inDailyDaily Status Report Status Report Issue Remains Issue Remains Closed Closed Test Plan is Test Plan is Updated to Reflect Updated to Reflect Issue as "not a Issue as "not a bug"
bug" (if (if applicableapplicable
Escalation Escalation Request? Request? Bug Annotated Bug Annotated in BTS in BTS Issue Issue Retested Retested Again (on Again (on Differnent Differnent combos / combos / Case Case Re-examined examined Still Still Exists? Exists? Bug Added to Bug Added to BTS BTS Bug ID Bug ID Entered in Entered in Defect Column Defect Column No No Bug Added to Bug Added to "Annotation" "Annotation" Report, Sent w/ Report, Sent w/ Daily Status Daily Status Report Report AppLabs to AppLabs to Test when Test when status status changed changed Escalation Escalation Added to Added toDailyDaily Status Report Status Report
Escalation Request will be Escalation Request will be made for 1 of 2 Reasons made for 1 of 2 Reasons (1) Tester believes issue is (1) Tester believes issue is more severe than it's current more severe than it's current status / fix date
status / fix date
(2) Issue is blocking testing (2) Issue is blocking testing of Key Functionality of Key Functionality Yes Yes Yes Yes Issue Issue Closed Closed No No Yes Yes Fixed? Fixed? Closed? Closed? Rejected? Rejected? Postponed? Postponed? Finished Finished No No No No Yes Yes Yes Yes "Not a "Not a Bug"? Bug"? Duplicate? Duplicate? Tester Tester Review & Review & Agree? Agree?
Bug Annotated & Bug Annotated & Bug Identified in Bug Identified in Daily Status Daily Status Report Report Cannot Cannot Reproduce? Reproduce? No No Client Client Agrees? Agrees? Bug Bug Re-Opened Opened Y Yeess YYeess No No Yes Yes No No Yes Yes No No Yes Yes No No Yes Yes No No No No Bug Bug Remains Remains Closed Closed Yes Yes No No "F" entered in "F" entered in Status Column Status Column Yes Yes Yes Yes 3.4 Severity Levels 3.4 Severity Levels
A severity level is supplied
A severity level is supplied for each problem for each problem issue identified. issue identified. AppLabs identifies five levels of AppLabs identifies five levels of severity.
severity.
Severity Level 1: Critical Severity Level 1: Critical
Program aborts and the system cannot function further Program aborts and the system cannot function further
Severity Level 2: High Severity Level 2: High
Major functionality of the application is performing improperly. Major functionality of the application is performing improperly. Severity Level 3: Medium
Severity Level 3: Medium A minor or
A minor or non-mission critical aspect of the appnon-mission critical aspect of the application is performing improperly. lication is performing improperly. TheThe existence of this error does not affect the users ability to access all crucial areas and existence of this error does not affect the users ability to access all crucial areas and functionality of the application.
functionality of the application. Severity Level 4: Low
Severity Level 4: Low There are
There are cosmetic or mcosmetic or minor usability issues. inor usability issues. The application functions properly The application functions properly even witheven with the existence of these issues.
the existence of these issues. Severity Level 5: Change Severity Level 5: Change These are change
These are change suggestions provided by AppLabs suggestions provided by AppLabs QA engineers. QA engineers. The application stillThe application still meets its functional and design requirements.
meets its functional and design requirements.
4.0 Fix Validation Process
4.0 Fix Validation Process
The fix validation process is aThe fix validation process is a critical component to success software development. critical component to success software development. To that end,To that end, AppLabs relies on a series of steps so issues reported by development as “fixed” are properly AppLabs relies on a series of steps so issues reported by development as “fixed” are properly handled.
handled. We retest each of these items, annotation item details within the bug tracker, and changeWe retest each of these items, annotation item details within the bug tracker, and change the issue’s status accordingly.
Query for "Fixed" Query for "Fixed"
Bugs Bugs
Generate Excel Generate Excel Table to List Bugs Table to List Bugs by Severity and by Severity and Appliction Section Appliction Section
Distribute Bugs for Distribute Bugs for
Review to Review to Appropiate Tester Appropiate Tester Tester Reviews Tester Reviews Result? Result? Closed Entered Closed Entered into the Excel into the Excel Spreadsheet Spreadsheet Bug Status Bug Status Updated to Closed Updated to Closed in BTS in BTS Date, Build #, Date, Build #, Tester added to Tester added to Resolution Field Resolution Field Details of Details of Spreadsheet sent Spreadsheet sent to Client in to Client inDailyDaily
Status Report Status Report
Open Entered into Open Entered into
the Excel the Excel Spreadsheet Spreadsheet Bug Status Bug Status Changed to Open Changed to Open in BTS in BTS Date, Build #, Date, Build #, Tester added to Tester added to Resolution Field Resolution Field Bug Status Bug Status Changed to Open Changed to Open in BTS in BTS Date, Build #, Date, Build #, Tester, New Tester, New results added in results added in Resolution Column Resolution Column Details of Details of Spreadsheet sent Spreadsheet sent to Client in to Client inDailyDaily
Status Report Status Report Details of Details of Spreadsheet sent Spreadsheet sent to Client in to Client inDailyDaily
Status Report Status Report Discrepency Discrepency Level (Old vs. Level (Old vs. New Issue) New Issue) Closed Entered Closed Entered into the Excel into the Excel Spreadsheet Spreadsheet
Open Entered into Open Entered into
the Excel the Excel Spreadsheet Spreadsheet
NEW Bug Entered NEW Bug Entered
into BTS into BTS Bug Status Bug Status Changed to Changed to Closed in BTS Closed in BTS Date, Build #, Date, Build #, Tester, New Bug Tester, New Bug Number added as Number added as Reference Reference Issue no longer Issue no longer Occuring Occuring Tester Obtains Tester Obtains Expected Result Expected Result Issue is ReOccuring Issue is ReOccuring Original Bug no longer Occuring,
Original Bug no longer Occuring, but Expected
but Expected Result not Result not obtainedobtained
SpeadSheet Filed SpeadSheet Filed by Date and Kept by Date and Kept
in Project's in Project's Directory Structure Directory Structure Details of Details of Spreadsheet sent Spreadsheet sent to Client in Daily to Client in Daily Status Report Status Report Low Low High High
5.0 Reporting
5.0 Reporting
AppLabs provides daily, weekly and end of cycle reporting throughout the entire QA cycle. AppLabs provides daily, weekly and end of cycle reporting throughout the entire QA cycle. 5.1 Daily Summary Data
5.1 Daily Summary Data
In addition to entering new bugs and closing fixed bug in the bug tracking application, AppLabs In addition to entering new bugs and closing fixed bug in the bug tracking application, AppLabs provides our clients the following detailed information.
provides our clients the following detailed information. •
• Number of test cases completedNumber of test cases completed •
• Number of new issues identified and reportedNumber of new issues identified and reported •
• Number of test cases, which failedNumber of test cases, which failed •
• Number of test cased blockedNumber of test cased blocked •
• The executed test plansThe executed test plans •
• The current list of existing bugs by severityThe current list of existing bugs by severity
•
• The test-tracking document. The test-tracking document. A sample A sample test tracking document test tracking document is showis show Test Tracking Document
Test Tracking Document
Date
Date Build Build ID ID Section Section Browser Browser NewNew tests tests against against this Build this Build Tests Tests repeated repeated against this against this Build Build Total Total tests run tests run this day this day Tests Tests Passed Passed Tests Tests Failed Failed Tests Tests Blocked Blocked New New Issues Issues Existing Existing Issues Issues 9 9--JJuul l 1 1 UUI I IIE E 44..001 1 117788 00 117788 111155 663 3 2288 1 100--JJuul l 1 1 UUI I IIE E 44..001 1 151544 00 115544 9944 660 0 6633 1 111--JJuul l 1 1 BBE E NNA A 2222 00 2222 88 114 4 1111 1 122--JJuul l 1 1 CCL L NNA A 2424 00 2244 1177 7 7 3377 1 133--JJuul l 1 1 BBE E NNA A 3333 00 3333 1144 119 9 3311 1 166--JJuul l 1 1 UI UI NNS S 66..001 1 181855 00 118855 113355 550 0 3377 1 177--JJuul l 1 1 UI UI NNS S 66..001 1 171744 00 117744 111100 664 4 3344 1 188--JJuul l 1 1 UI UI IIE E 55..0 0 222299 00 222299 141466 883 3 2266 1 199--JJuul l 2 2 UUI I IIE E 44..001 1 223300 00 223300 191966 334 4 2211 2 200--JJuul l 2 2 UUI I IIE E 44..001 1 226666 00 226666 117788 88 88 2211 33 3399 2 233--JJuul l 2 2 BBE E / / CCL L NNA A 7070 00 7700 5566 114 4 2233 66 00 2 244--JJuull 22 BBEE//CCLL NNAA 5533 00 5533 5500 33 11 11 22 2 255--JJuull 22 BBEE//CCLL NNAA 3311 00 3311 3300 11 00 00 11 2 266--JJuul l 2 2 UUI I NNS S 44..008 8 227755 00 227755 224433 332 2 44 00 1122
5.2 Weekly Summary Report 5.2 Weekly Summary Report
At the end of each week, AppLabs creates a weekly report to provide our clients additional At the end of each week, AppLabs creates a weekly report to provide our clients additional information.
information. The weekly The weekly report contains bug report contains bug information by test information by test area. area. A section A section of a of a sample reportsample report is shown below.
is shown below.
Login / Registration Login / Registration Outstanding
Outstanding Week Week ReviewReview
Hig
High h 55 CloClosed sed / / FixFixed ed 88
Me
Medidium um 44 NeNew w 11
9 9 Create New Patient
Create New Patient Outstanding
Outstanding Week Week ReviewReview
Hig
High h 11 CloClosed sed / / InvInvalialid d 22 Ch
Chanange ge 11 NeNew w 22
Low 1 Low 1 Block 1 Block 1 4 4 Call in Patient Call in Patient Outstanding
Outstanding Week Week ReviewReview
Hig
High h 11 CloClosed sed / / FixFixed ed 11
Mediu
Medium m 11 CloseClosed d / / DuplicDuplicate ate 11 Low
Low 44 CloClosed sed /Fi/Fixed xed 44
Ch
Chanange ge 33 NeNew w 33
9 9 Pre-Triage Patient
Pre-Triage Patient Outstanding
Outstanding Week Week ReviewReview
Med
Medium ium 11 CloClosed sed / / FixFixed ed 11 Ch
Chanange ge 33 NeNew w 33
4 4 Triage Patient
Triage Patient Outstanding
Outstanding Week Week ReviewReview
Sev
Severe ere 22 CloClosed sed / / FixFixed ed 1010 Hig
High h 33 CloClosed sed /In/Invalvalid id 11 Me
Medidium um 22 NeNew w 1919
Low 16 Low 16 Change 8 Change 8 31 31
5.3 End of Cycle Reports 5.3 End of Cycle Reports
When the AppLabs QA engineers execute all test cases for a build, we supply our clients a When the AppLabs QA engineers execute all test cases for a build, we supply our clients a complete set of
complete set of test results for the test results for the cycle. cycle. We provide the We provide the following informatiofollowing information:n: •
• Each completed test planEach completed test plan •
• List of bugs found in this testing cycleList of bugs found in this testing cycle •
• List of bugs fixed in this testing cycleList of bugs fixed in this testing cycle •
• List of bugs deferred to a later releaseList of bugs deferred to a later release
Full test tracking document for the completed cycle Full test tracking document for the completed cycle
6.0 Compatibility Test
6.0 Compatibility Test
AppLabs performs functionalAppLabs performs functional testing on testing on a variety of a variety of platform/browser combinations. platform/browser combinations. The testingThe testing combinations will be identified during the scope process.
combinations will be identified during the scope process. 6.1 Browser Checks
6.1 Browser Checks
AppLabs can perform cross-browser compatibly tests to ensure applications can perform properly AppLabs can perform cross-browser compatibly tests to ensure applications can perform properly on different browsers.
on different browsers. We will adjust the machine configurations within our QA We will adjust the machine configurations within our QA lab to meet our lab to meet our client’s application browser requirements.
client’s application browser requirements. Sample Browser List
Sample Browser List 1. 1. I.E I.E 4.04.0 2. 2. I.E I.E 4.54.5 3. 3. I.E I.E 5.05.0 4. 4. I.E. I.E. 5.55.5 5. 5. N.S N.S 4.54.5 6. 6. N.S N.S 4.754.75 7. 7. N.S N.S 6.06.0 8. 8. N.S. N.S. 6.16.1 6.2 Operating Systems 6.2 Operating Systems
AppLabs maintains multiple operating systems on
AppLabs maintains multiple operating systems on which to perform which to perform testing. testing. Based on Based on the client’sthe client’s requirements we can perform the functional testing on these operating systems in various
requirements we can perform the functional testing on these operating systems in various combinations with the browsers above.
combinations with the browsers above.
1 Linux
1 Linux
2
2 Solaris Solaris 2.6, 2.6, 5.6, 5.6, 7.2, 7.2, 8.08.0 3
3 Various Various Windows Windows Versions Versions including: including: Win Win 95,95, Win 98, Win NT, Win ME, Win 2000
Win 98, Win NT, Win ME, Win 2000 4
4 Multiple Multiple MAC MAC Versions Versions including including Mac Mac 8.6,8.6, Mac 9, Mac 10
Mac 9, Mac 10
7.0 Automated Testing
7.0 Automated Testing
AppLabs utilizes Mercury WinRunner to develop
AppLabs utilizes Mercury WinRunner to develop automated test scripts. automated test scripts. Scripts are developed andScripts are developed and updated by
updated by AppLabs. AppLabs. The depth The depth and breath and breath of utilizing automated of utilizing automated testing will be testing will be set at set at thethe beginning of the project.
beginning of the project. As the QA As the QA project progresses and the product becomes more stable,project progresses and the product becomes more stable, there are additional areas where
there are additional areas where automated testing may automated testing may be beneficial. be beneficial. AppLabs and AppLabs and our clientsour clients maintain a constant dialog concerning the potential deployment of automated testing.
maintain a constant dialog concerning the potential deployment of automated testing.
8.0 Stress, Load and Capacity Testing
8.0 Stress, Load and Capacity Testing
AppLabs utilizes Mercury Interactive’s LoadRunner to
AppLabs utilizes Mercury Interactive’s LoadRunner to conduct stress, load conduct stress, load and capacity and capacity tests. tests. TheThe details of our approach can be found in the AppLabs QA Stress, Load and Capacity Testing
details of our approach can be found in the AppLabs QA Stress, Load and Capacity Testing document.
8.1 Load Testing 8.1 Load Testing
Testing an application against a
Testing an application against a requested number of requested number of users. users. TheThe objectiveobjective is to determine whether is to determine whether the site can sustain this requested number of users with acceptable response times.
the site can sustain this requested number of users with acceptable response times. 8.2 Stress Testing
8.2 Stress Testing Load testing over
Load testing over an extended period an extended period of time. of time. TheThe objectiveobjectiveis to validate an application’s stabilityis to validate an application’s stability and reliability.
and reliability.
8.3 Capacity Testing 8.3 Capacity Testing
Testing to determine the maximum
Testing to determine the maximum number of concurrent users an application can manage. number of concurrent users an application can manage. TheThe
objective
objectiveis to benchmark the maximum loads of concurrent users a site can sustain beforeis to benchmark the maximum loads of concurrent users a site can sustain before experiencing system failure.
experiencing system failure. 8.4 Reporting Terms
8.4 Reporting Terms
Report information containing the following information is provided to or clients. Report information containing the following information is provided to or clients.
Load size Load size
This is the number of concurrent Virtual Clients trying to access the site. This is the number of concurrent Virtual Clients trying to access the site. Throughput
Throughput
The average number of bytes per second transmitted from the ABT (Application being The average number of bytes per second transmitted from the ABT (Application being tested) to the Virtual Clients running this Agenda during the last reporting interval tested) to the Virtual Clients running this Agenda during the last reporting interval Round Time
Round Time
It is the average time it took the virtual clients to finish one complete iteration of the agenda It is the average time it took the virtual clients to finish one complete iteration of the agenda during the last reporting interval.
during the last reporting interval. Transaction Time
Transaction Time
The time it takes to complete a successful HTTP request, in seconds. (Each request for The time it takes to complete a successful HTTP request, in seconds. (Each request for each gif, jpeg, html file, etc. is a single transaction.) The time of a transaction is the sum of each gif, jpeg, html file, etc. is a single transaction.) The time of a transaction is the sum of the Connect Time, Send Time, Response Time, and Process Time.
the Connect Time, Send Time, Response Time, and Process Time. Connect Time
Connect Time
The Time it takes for a Virtual client to connect to the Application Being Tested. The Time it takes for a Virtual client to connect to the Application Being Tested. Send Time
Send Time
The time it takes the Virtual Clients to write an HTTP request to the ABT (Application being The time it takes the Virtual Clients to write an HTTP request to the ABT (Application being tested), in seconds.
tested), in seconds. Response Time Response Time
The time it takes the ABT (Application being tested) to send the object of an HTTP request The time it takes the ABT (Application being tested) to send the object of an HTTP request back to a Virtual Client, in seconds. In other words, the time from the end of the HTTP back to a Virtual Client, in seconds. In other words, the time from the end of the HTTP request until the Virtual Client has received the complete item it requested.
request until the Virtual Client has received the complete item it requested. Process Time
Process Time
The time it takes WebLoad to parse an HTTP response from the ABT (Application being The time it takes WebLoad to parse an HTTP response from the ABT (Application being tested) and then populate the document-object model (the DOM), in seconds.
tested) and then populate the document-object model (the DOM), in seconds. Wait Time (Average Latency)
Wait Time (Average Latency)
The time it takes from when a request is sent until the first byte is received. The time it takes from when a request is sent until the first byte is received.
This is the elapsed time between receiving the first byte and the last byte. This is the elapsed time between receiving the first byte and the last byte.
9.0 Minimal Acceptance Tests
9.0 Minimal Acceptance Tests
Once the entire testing cycle has been completed and the application is being considered to be Once the entire testing cycle has been completed and the application is being considered to be moved out of the
moved out of the QA environment, a final set of QA environment, a final set of pre-defined tests should be conducted. pre-defined tests should be conducted. This will beThis will be a sub-set of the detailed test scenarios developed from each area of the test plan. The results of the a sub-set of the detailed test scenarios developed from each area of the test plan. The results of the acceptance test will determine whether the application has passed the QA exit criteria and is ready acceptance test will determine whether the application has passed the QA exit criteria and is ready to be deployed to a staging or live environment.
to be deployed to a staging or live environment.
10.0 Communication
10.0 Communication
Ongoing communication is conducted
Ongoing communication is conducted between our dedicated between our dedicated QA team QA team and our clients. and our clients. TheThe communications mechanisms employed by our QA unit are listed below.
communications mechanisms employed by our QA unit are listed below. •
• Daily EmailsDaily Emails •
• Weekly Conference CallsWeekly Conference Calls •
• Weekly yahoo or other online chat sessionsWeekly yahoo or other online chat sessions •
• Weekly status reports containing entire weeks completed work and projections for upcomingWeekly status reports containing entire weeks completed work and projections for upcoming week
week •
• End of Cycle ReportsEnd of Cycle Reports •
