Windows with Microsoft SQL Server Performance Load
Testing on Clarity 7.5.1
Technical White Paper
Version: 2.0
October 20, 2005
Table of Contents
EXECUTIVE SUMMARY... 3
TEST ENVIRONMENT... 4
TEST RESULT SUMMARIES... 6
APPENDIX A: DETAILED LOADRUNNER TEST SCRIPTS... 7
APPENDIX B: USER RIGHTS ... 8
APPENDIX C: MEDIUM USER BASE TEST DETAILED RESULTS ... 15
APPENDIX D: SMALL USER BASE TEST DETAILED RESULTS... 17
Executive Summary
Introduction
In order to establish and publish performance benchmarks, Computer Associates Corporation conducted load tests of Clarity 7.5.1 on medium and small scale Windows environments. The objectives of these tests were to:
• Demonstrate Clarity’s ability to serve small and medium user bases (1,000 to 2,000 users) on Intel based hardware using a Microsoft SQL database and Orion application servers, and to
• Establish performance benchmarks under load using a core set of application functionality on Intel based hardware and Windows environment.
The load tests used a mixture of read and write actions to simulate user interactions. A concurrency rate of 20% was used, with an average wait time of 60 seconds between page requests; for the purposes of our load testing, the average wait time was accelerated to 15 seconds. The performance was measured for 30 minutes, excluding a ramp-up time of 15 minutes.
The user-base tests used a single database server with two load balanced application servers. The actual experience of customers will depend on how their hardware is set up and how many Clarity modules they deploy.
Results and Conclusion
Clarity 7.5.1 can adequately support small user bases and medium user bases on dual (load balanced) application server Windows configurations. See the Test Environment section of this document for more information about test set-up. See Appendices C and D for detailed test results.
Response times for the ninety percent of user activity typically measure less than 0.8 seconds.
Measurement Small Load
Windows
Medium Load Windows
Average transaction response time 0.366 seconds 0.4 seconds 90th percentile response time 0.481 seconds 0.531 seconds
Average Throughput 67 Kbs 124 Kbs
Average Database server CPU utilization 2.46 % 4.79 % Average Application server CPU utilization 7.58 % 15.41 %
Disclaimer
This white paper is not meant as a sizing guide or as a recommendation guide but instead portrays possible user loads given a particular Clarity configuration. We understand that customer requirements and environments differ. The Clarity functionality exercised during these performance load tests may or may not match the functionality that individual companies deploy. Computer Associates does not recommend that customers acquire hardware based on this white paper but instead seek the sizing expertise of our technical pre-sales and professional services personnel. The test environments outlined in this document do not consider reporting, process engine, and background service or failover requirements.
Test Environment
Test Overview
• All tests were written using Mercury’s LoadRunner tool set.
• Appendix A contains the detailed test scenario descriptions. Identical LoadRunner test scripts were run for all load tests (in the same ratios; see below).
• One Virtual User represents 20 real users (i.e. uses a 20% concurrency and an accelerated think time of 15 seconds, representing a normalized “real world” think time of 60 seconds).
• All runs were 30 minutes long, excluding ramp up/down time (approx 15 minutes).
Test Scenarios
The test scenario reflects a realistic mixture of User activities on the system. We use a combination of 5 navigation scripts and a timesheet creation script in the following ratios:
Scenario Weighting
General navigation. 30%
Timesheet navigation. 25%
Project navigation. 20%
Resource navigation 20%
Timesheet creation 5%
User Simulation
• Small – this corresponds to the average Customer deployment size.
• Medium – this corresponds to the average Customer deployment size.
Size Platform Number of VUsers Registered Users
Small Windows/MSSQL 50 1,000
Medium Windows/MSSQL 100 2,000
Test Data Set
The data used for these tests was generated by Computer Associates; customer data was not used. The data was loaded by XOG and through the User Interface via Mercury WinRunner scripts. Because we used specific users for each test (and avoided using global rights), no users could see all the data elements. The data used for these tests is as follows:
Element Quantity Notes
Resources 5366 Projects 579 Timesheets 20,472
Timesheet reporting periods 53 Only 4 were open
Portfolios 5 Programs 8 Ideas 9
User Base Configuration
The user-base test runs simulated the load that can be expected for customers with small and medium systems. The hardware to support this consists of two load-balanced application servers each running two application server instances.
The table in the following diagram shows the details for the Windows environment. No SSL or LDAP services were running.
Performance Environment
Hardware Configuration
Load Balancer 1 Foundry Networks ServerIron XL 24 Port Load Balancing Switch
Switch Catalyst 2900
Windows AppServers
SuperMicro, Win 2003, Dual 2.8Ghz, 4Gb RAM. Application and background services.
MS SQL 2000 Database Server
SuperMicro, Win 2003, Dual 2.8Ghz, 4Gb RAM. DB service.
LoadRunner Controller
SuperMicro, Win 2003, Dual 2.8Ghz, 2Gb RAM
LoadRunner Agents
Win 2K, 663 MHz, 260K RAM
Test Result Summaries
Medium User Base Results
Two SuperMicro application servers, both running two Clarity 7.5.1 FP02 application instances (1.5 GB JVMs), can support a simulated load of 2,000 registered users (100 virtual users).
• Average transaction response time: 0.4 seconds
• 90th percentile response time: 0.531 seconds
• Average Throughput: 124 KB per second
• Average Database server utilization: 4.79 %
• Average Application server utilization: 15.41 % See Appendix C for more detailed results.
Small User Base Results
One SuperMicro application server running a Clarity 7.5.1 FP02 application instance (1.5 GB JVM) can support a simulated load of 1,000 registered users (50 virtual users).
• Average transaction response time: 0.366 seconds
• 90th percentile response time: 0.481 seconds
• Average Throughput: 67 KB per second
• Average Database server utilization: 2.46 %
• Average Application server utilization: 7.58 % See Appendix D for more detailed results.
Appendix A: Detailed LoadRunner Test Scripts
To minimize excessive log in and log out operations, virtual user scripts will iterate the login steps from the main script 2-3 times after log in. After the iterations complete, the virtual user will log out and the next virtual user will start. In other words, there is an inner loop between log in and log out.
Navigation all links in the left pane: • Log in as the link_viewer user.
• Click everything on the left navigation frame (Overview, Organizers, Report and Jobs, Account Settings,
Portfolios, Programs, Project, Applications, Products, Idea).
• Log out.
Timesheet Navigations: • Login as perfuser01-59.
• User click timesheet link on personal view.
• Filtering for open timesheets (page down/up).
• Click first open timesheet.
• Click task.
• Click cancel.
• Log out.
Projects navigation:
• Login as project_view.
• Click the projects link on the navigation frame.
• Click sort ascending and descending on project name column.
• Click sort ascending and descending on project ID column.
• Click to open the first project from the list.
• Click the project properties, team, tasks and process tabs.
• Log out.
Resources navigations:
• Log in as resource_viewer.
• Click on the resource link.
• Click sort ascending and descending on resource name column.
• Click sort ascending and descending on resource ID column.
• Click to open the first resource from the list (resource_properties).
• Click Project tab.
• Click Calendar tab.
• Log out.
Timesheet entry:
• Log in as perfuser01-50.
• Open the first time sheet.
• Enter times for 1 task.
• Save.
• Submit for approval.
• Open the submitted time sheet
• Click return timesheet button.
• Log out.
Appendix B: User Rights
The test users performing the test scenarios described in Appendix A above were given the following rights:
Link_View User Instance Rights:
Application - Approve Application Allows approval of an Application
Application - Edit Application Allows editing an application, includes delete Application - Edit
Access
Application Allows editing of Application access rights page Application - View Application Allows viewing an application
Asset - Approve Asset Allows approval of an Asset
Asset - Edit Asset Allows editing an asset, includes delete
Asset - Edit Access Asset Allows editing of Asset access rights page
Asset - View Asset Allows viewing an asset
Job - Run Job/Report Run a specific job. Allows editing of the scheduled job properties and viewing of the output of the job. This right is dependent on Jobs - Access being granted.
Other Investment - Approve
Other Investment Allows approval of an Other Investment Other Investment -
Edit
Other Investment Allows editing an other investment, includes delete Other Investment -
Edit Access
Other Investment Allows editing of Other Investment access rights page Other Investment -
View
Other Investment Allows viewing an other investment
Portfolio - Edit Portfolio Allows editing of a portfolio, includes delete Portfolio - Edit Access Portfolio Allows editing of Portfolio access rights page Portfolio - Scenario
Create
Portfolio Allows creation of Scenarios for a Portfolio Portfolio - View Portfolio Allows viewing of a portfolio
Portfolio - View All Scenarios
Portfolio Allows viewing of all scenarios for a Portfolio Product - Approve Product Allows approval of a product
Product - Edit
Product - Edit Access Product Allows editing of Product access rights page
Product - View Product Allows viewing a product
Project - Approve Project Allows approval of a Project Project - Attach
Requisition Entry Resources
Project Allows resource to attach resources to requisition entries. This right includes the right to navigate to the requisition pages.
Project - Create - sub_project
Project Allows resource to create sub_project sub objects within a specific Project master object.
Project - Create - sub_sub_project
Project Allows resource to create sub_sub_project sub objects within a specific Project master object.
Project - Create/Edit Requisitions
Project Allows resource to create and edit requisition properties. This right includes the right to navigate to the requisition pages, but not to create or edit the entries within the requisitions.
Project - Delete Project Allows resource to delete a project or program.
Project - Edit Project Allows resource to edit all parts of a project except for Document Management, Calendar, Action Items, Discussions and Custom Defined Fields.
Project - Edit Access Rights
Project Allows resource to manage access rights for a project. This right is dependent on Project - Edit Management being granted for the project.
Project - Edit All - Project Allows resource to edit all sub_project sub objects within a specific
sub_project Project master object. Project - Edit All -
sub_sub_project
Project Allows resource to edit all sub_sub_project sub objects within a specific Project master object.
Project - Edit Assigned Tasks
Project Edit Assigned Tasks on a Project
Project - Edit Financial Project Allows resource to edit the general and management properties, roster and key tasks. The resource can also add sub-projects to the project and the project can be viewed and edited in Microsoft Project or Open Workbench.
Project - Edit Management
Project Allows resource to edit the general and management properties, staff and tasks for the selected project if it has been enabled for management. This includes the ability to add sub-projects to it as well as edit it in Microsoft Project and Open Workbench.
Project - Edit Opportunity
Project Allows resource to edit opportunity and general properties of the project. This right implies that the opportunity module for the project has been enabled.
Project - Edit Project Plan
Project Allows resource to add unplanned tasks to the selected project when completing his timesheet if he is a team member on that project.
Project - Edit Requisition Entries
Project Allows resource to create and edit requisition entries. This right includes the right to navigate to the requisition pages.
Project - Financial Planning
Project Allows resource to view and edit the financial plan for the projects. This allows the resource to enter budgets and forecasts and view the general project properties.
Project - Financial Planning - Budget
Project Allows resource to view and edit the financial budget plan for the projects. This allows the resource to enter budgets and view the general project properties
Project - Financial Planning - Forecast
Project Allows resource to view and edit the financial forecast plan for the projects. This allows the resource to enter forecasts and view the general project properties.
Project - Financial Planning Approval
Project Allows resource to approve and submit the financial plan for the project. This right allows resource to enter budgets and forecasts and view the general project properties.
Project - Financial Planning Approval - Budget
Project Allows resource to approve and submit the financial budget plan for the project. This right allows resource to enter budgets and view the general project properties.
Project - Financial Planning Approval- Forecast
Project Allows resource to approve and submit the financial forecast plan for the project. This right allows resource to enter forecasts and view the general project properties.
Project - Modify Baseline
Project Allows resource to baseline a project if it has been enabled for Management. The rights also allows resource to view the general and management properties and processes.
Project - Risk, Issue, Change Request - Create/Edit
Project Allows resource to create and edit risks, issues, and changes for a project or program instance on which they are a staffed.
Project - Risk, Issue, Change Request - Delete
Project Allows resource to delete risks, issues, and changes for a project or program instance on which they are a staffed.
Project - Risk, Issue, Change Request - View
Project Allows resource to view risks, issues, and changes for a project or program instance on which they are staffed.
Project - View Project Allows resource to view the general, management, financial and opportunity properties, custom defined fields, roster, tasks, processes and sub-projects for the project.
Project - View Access Rights
Project Allows resource to view access rights for a project. From within the application, this right implies that the resource has Project - View right to the project also. For a resource trying to view the access rights within the Administration section, this right is dependent on the having Resource - Edit Administration granted.
Project - View All - sub_project
Project Allows resource to view all sub_project sub objects within a specific Project master object.
Project - View All - sub_sub_project
Project Allows resource to view all sub_sub_project sub objects within a specific Project master object.
Project - View CDF Project Allows resource to view the general properties and custom defined fields for a project.
Project - View Documents
Project Allows resource to view documents or folders even if resource is not a project participant, as long as the documents or folders are marked open to non-participants. This right includes the Project - View Base right.
Project - View Financial
Project Allows resource to view the general and financial properties for the project.
Project - View Management
Project Allows resource to view management properties, roster and key tasks of the project. This right also allows resource view access to the project in Microsoft Project and Niku Workbench.
Project - View Opportunity
Project Allows resource to view the general and opportunity properties for the project.
Project - View Requisition Entries
Project Allows resource to view requisition entries. This right includes the right to navigate to the requisition pages.
Project - View Requisitions
Project Allows resource to view a requisition. This right is dependent on Resource - External Access being granted.
Project - View Tasks Project Allows resource to view tasks for a project. This right is dependent on Project - View Base being granted.
Report - Run Job/Report Allows resource to run a specific report. The right also allows editing of the scheduled report properties and viewing of the output of the report. This right is dependent on Reports - Access being granted.
Resource - Approve Ideas
Resource Allows approval of ideas for individual resources Resource - Approve
Time
Resource Allows the resource to approve and reject timesheets for a resource. The right does not imply Resource - Enter Time. The Timesheets link will appear on the personal page.
Resource - Edit Resource Allows the resource to view and edit all properties for a resource. This right is dependent on Resource - Navigate being granted. Resource - Edit
Access Rights
Resource Allows resource to grant or revoke access rights for a resource. This right is dependent on Resource - View or Resource - View - All being granted to access the page from within the Application. To access the page from within Administration section, this right is dependent on Resource - Edit Administration being granted. Resource - Edit
Administration
Resource Allows resource to update authentication and authorization information for a resource. This right is dependent on Resource - Authorization Access Screens being granted.
Resource - Edit Calendar
Resource Allows resource to edit their own calendar. This right is dependent on Resource - Navigate being granted.
Resource - Edit Ideas Resource Allows editing of ideas for individual resources, includes delete Resource - Enter Time Resource Allows resource to complete and submit and reject timesheets for a
resource. The Timesheets link will appear on the personal page. Resource - Hard Book Resource Allows resource to soft- or hard book a resource to a project to
which he has the Project - View Management right.
Resource - Self Resource Rights automatically granted to a resource when created. Includes Resource - Edit Ideas.
Resource - Soft Book Resource Allows resource to soft-book a resource to a project to which he has the Project - View Management right. The right also allows the resource to edit any soft-booking for the specified resource (or to remove it from the project).
Resource - Update Skills
Resource Allows resource to update the skills profile for a resource as well as to modify the general resource properties.
Resource - View Resource Allows resource to view general information, contact information,
financial information, custom fields, skills, project list and working calendar for a resource.
Resource - View Access Rights
Resource Allows resource to view access rights for a resource. From within the Application this right is dependent on Resource - View being granted. From within the Administration section this right is dependent on Resource - Edit Administration being granted. Resource - View Book Resource Allows resource to view bookings for a resource.
Resource - View Financial
Resource Allows resource to view financial information for a resource. Resource - View Ideas Resource Allows viewing of ideas for individual resources
Link_View User Global Rights:
Application - Navigate Access to the application management pages Resource
Asset - Navigate Access to the asset management pages Resource
Ideas - Navigate Access to the ideas management pages Resource
Jobs - Access Allows resource access to the jobs pages. This right is dependent on either the Jobs - Run - All right or the resource being granted instance level rights such as Job - Run, Job - View Output or Job - Edit Properties.
Resource
Management - Programs
Allowed resource access to the Program Management features for
programs and projects the resource has access to. This right is dependent on the resource having rights to programs and projects at either the instance level or OBS level.
Resource
Other Investment - Navigate
Access to the other investment management pages Resource
Portfolio - Navigate Access to the portfolio management pages Resource
Product - Navigate Access to the product management pages Resource
Reports - Access Allows resource access to the reports pages. This right is dependent on either the Reports - Run - All right or the resource being granted instance level rights such as Report - Run, Report - View Output or Report - Edit Properties.
Resource
Resource - Navigate Allows resource access to the Resource list, Resource Finder, and Requisitions pages. The resource will only be able to view resource information for resources to which the resource has access.
Resource
Timesheets - Navigate Allows resource to access the timesheets list page. The resource will only be able to see timesheets the resource has edit or approve timesheets rights over.
Resource
Project_View User Instance Rights:
Project - Approve Project Allows approval of a Project Project - Attach
Requisition Entry Resources
Project Allows resource to attach resources to requisition entries. This right includes the right to navigate to the requisition pages.
Project - Create - sub_project
Project Allows resource to create sub_project sub objects within a specific Project master object.
Project - Create - sub_sub_project
Project Allows resource to create sub_sub_project sub objects within a specific Project master object.
Project - Create/Edit Requisitions
Project Allows resource to create and edit requisition properties. This right includes the right to navigate to the requisition pages, but not to create or edit the entries within the requisitions.
Project - Delete Project Allows resource to delete a project or program.
Project - Edit Project Allows resource to edit all parts of a project except for Document Management, Calendar, Action Items, Discussions and Custom Defined Fields.
Project - Edit Access Rights
Project Allows resource to manage access rights for a project. This right is dependent on Project - Edit Management being granted for the project. Project - Edit All - Project Allows resource to edit all sub_project sub objects within a specific
sub_project Project master object. Project - Edit All -
sub_sub_project
Project Allows resource to edit all sub_sub_project sub objects within a specific Project master object.
Project - Edit Assigned Tasks
Project Edit Assigned Tasks on a Project Project - Edit
Financial
Project Allows resource to edit the general and management properties, roster and key tasks. The resource can also add sub-projects to the project and the project can be viewed and edited in Microsoft Project or Open Workbench.
Project - Edit Management
Project Allows resource to edit the general and management properties, staff and tasks for the selected project if it has been enabled for
management. This includes the ability to add sub-projects to it as well as edit it in Microsoft Project and Open Workbench.
Project - Edit Opportunity
Project Allows resource to edit opportunity and general properties of the project. This right implies that the opportunity module for the project has been enabled.
Project - Edit Project Plan
Project Allows resource to add unplanned tasks to the selected project when completing his timesheet if he is a team member on that project. Project - Edit
Requisition Entries
Project Allows resource to create and edit requisition entries. This right includes the right to navigate to the requisition pages.
Project - Financial Planning
Project Allows resource to view and edit the financial plan for the projects. This allows the resource to enter budgets and forecasts and view the general project properties.
Project - Financial Planning - Budget
Project Allows resource to view and edit the financial budget plan for the projects. This allows the resource to enter budgets and view the general project properties.
Project - Financial Planning - Forecast
Project Allows resource to view and edit the financial forecast plan for the projects. This allows the resource to enter forecasts and view the general project properties.
Project - Financial Planning Approval
Project Allows resource to approve and submit the financial plan for the project. This right allows resource to enter budgets and forecasts and view the general project properties.
Project - Financial Planning Approval - Budget
Project Allows resource to approve and submit the financial budget plan for the project. This right allows resource to enter budgets and view the general project properties.
Project - Financial Planning Approval- Forecast
Project Allows resource to approve and submit the financial forecast plan for the project. This right allows resource to enter forecasts and view the general project properties.
Project - Modify Baseline
Project Allows resource to baseline a project if it has been enabled for Management. The rights also allows resource to view the general and management properties and processes.
Project - Risk, Issue, Change Request - Create/Edit
Project Allows resource to create and edit risks, issues, and changes for a project or program instance on which they are a staffed.
Project - Risk, Issue, Change Request - Delete
Project Allows resource to delete risks, issues, and changes for a project or program instance on which they are a staffed.
Project - Risk, Issue, Change Request - View
Project Allows resource to view risks, issues, and changes for a project or program instance on which they are staffed.
Project - View Project Allows resource to view the general, management, financial and opportunity properties, custom defined fields, roster, tasks, processes and sub-projects for the project.
Project - View Access Rights
Project Allows resource to view access rights for a project. From within the application, this right implies that the resource has Project - View right to the project also. For a resource trying to view the access rights within the Administration section, this right is dependent on the having
Resource - Edit Administration granted. Project - View All -
sub_project
Project Allows resource to view all sub_project sub objects within a specific Project master object.
Project - View All - sub_sub_project
Project Allows resource to view all sub_sub_project sub objects within a specific Project master object.
Project - View CDF Project Allows resource to view the general properties and custom defined fields for a project.
Project - View Documents
Project Allows resource to view documents or folders even if resource is not a project participant, as long as the documents or folders are marked open to non-participants. This right includes the Project - View Base right. Project - View
Financial
Project Allows resource to view the general and financial properties for the project.
Project - View Management
Project Allows resource to view management properties, roster and key tasks of the project. This right also allows resource view access to the project in Microsoft Project and Niku Workbench.
Project - View Opportunity
Project Allows resource to view the general and opportunity properties for the project.
Project - View Requisition Entries
Project Allows resource to view requisition entries. This right includes the right to navigate to the requisition pages.
Project - View Requisitions
Project Allows resource to view a requisition. This right is dependent on Resource - External Access being granted.
Project - View Tasks Project Allows resource to view tasks for a project. This right is dependent on Project - View Base being granted.
Resource - Enter Time
Resource Allows resource to complete and submit and reject timesheets for a resource. The Timesheets link will appear on the personal page. Resource - Self Resource Rights automatically granted to a resource when created. Includes
Resource - Edit Ideas.
Resource_Viewer Instance Rights:
Resource - Approve Ideas
Resource Allows approval of ideas for individual resources Resource - Approve
Time
Resource Allows the resource to approve and reject timesheets for a resource. The right does not imply Resource - Enter Time. The Timesheets link will appear on the personal page.
Resource - Edit Resource Allows the resource to view and edit all properties for a resource. This right is dependent on Resource - Navigate being granted.
Resource - Edit Access Rights
Resource Allows resource to grant or revoke access rights for a resource. This right is dependent on Resource - View or Resource - View - All being granted to access the page from within the Application. To access the page from within Administration section, this right is dependent on Resource - Edit Administration being granted.
Resource - Edit Administration
Resource Allows resource to update authentication and authorization information for a resource. This right is dependent on Resource - Authorization Access Screens being granted.
Resource - Edit Calendar
Resource Allows resource to edit their own calendar. This right is dependent on Resource - Navigate being granted.
Resource - Edit Ideas Resource Allows editing of ideas for individual resources, includes delete Resource - Enter
Time
Resource Allows resource to complete and submit and reject timesheets for a resource. The Timesheets link will appear on the personal page. Resource - Hard
Book
Resource Allows resource to soft- or hard book a resource to a project to which he has the Project - View Management right.
Resource - Self Resource Rights automatically granted to a resource when created. Includes Resource - Edit Ideas.
Resource - Soft Book Resource Allows resource to soft-book a resource to a project to which he has the Project - View Management right. The right also allows the resource to edit any soft-booking for the specified resource (or to remove it from the
project). Resource - Update
Skills
Resource Allows resource to update the skills profile for a resource as well as to modify the general resource properties.
Resource - View Resource Allows resource to view general information, contact information, financial information, custom fields, skills, project list and working calendar for a resource.
Resource - View Access Rights
Resource Allows resource to view access rights for a resource. From within the Application this right is dependent on Resource - View being granted. From within the Administration section this right is dependent on Resource - Edit Administration being granted.
Resource - View Book
Resource Allows resource to view bookings for a resource. Resource - View
Financial
Resource Allows resource to view financial information for a resource. Resource - View
Ideas
Resource Allows viewing of ideas for individual resources
Resource_Viewer Global Right:
Resource - Navigate Allows resource access to the Resource list, Resource Finder, and Requisitions pages. The resource will only be able to view resource information for resources to which the resource has access.
Resource
Appendix C: Medium User Base Test Detailed Results
Below are the detailed Mercury LoadRunner results for the Medium User Base test.
Action Description
Average Response
Time
90th Percentile
Number of Tests Run
TRS_ENTERTIME_00_login Time enter user login 0.797 1.008 59
TRS_ENTERTIME_00_logout Time enter user log out 0.277 0.384 59
TRS_ENTERTIME_00_startup Time enter script start session 0.071 0.123 5
TRS_ENTERTIME_01_timeperiod_ascending Sorting time periods ascend 0.192 0.414 59
TRS_ENTERTIME_01_timesheetlink Click time sheet link 0.183 0.273 59
TRS_ENTERTIME_02_opentimesheet Open a time sheet 0.265 0.404 59
TRS_ENTERTIME_02_savetimesheet Save time sheet 0.41 0.514 59
TRS_ENTERTIME_03_submitTS Submit Time Sheet 0.854 1.008 59
TRS_ENTERTIME_04_listsubmittedTS List submitted time sheet 0.118 0.153 59
TRS_ENTERTIME_04_opensubmittedTS Open submitted Time sheet 0.203 0.324 59
TRS_ENTERTIME_04_returnTS Return Time Sheet 0.348 0.494 59
TRS_NAVIGENERAL_00_login General navigation user login 0.771 0.847 30
TRS_NAVIGENERAL_00_logout General navigation user logout 0.232 0.253 30
TRS_NAVIGENERAL_00_startup General navigation session startup 0.062 0.073 30
TRS_NAVIGENERAL_01_AcctSettings Account settings link 0.085 0.123 276
TRS_NAVIGENERAL_01_Organizer Organizer link 0.188 0.253 276
TRS_NAVIGENERAL_01_Overview Overview page 0.361 0.484 276
TRS_NAVIGENERAL_01_ReportsJobs Reports and Jobs link 0.186 0.263 276
TRS_NAVIGENERAL_02_Applications Applications link 0.319 0.444 276
TRS_NAVIGENERAL_02_Assests Assests link 0.386 0.524 276
TRS_NAVIGENERAL_02_Ideas Ideas link 0.221 0.303 276
TRS_NAVIGENERAL_02_OtherInvests Other investments link 0.343 0.474 276
TRS_NAVIGENERAL_02_Portfolios Portfolios link 0.341 0.454 276
TRS_NAVIGENERAL_02_Products Products link 0.322 0.444 276
TRS_NAVIGENERAL_02_Programs Programs link 0.458 0.595 276
TRS_NAVIGENERAL_02_Projects Projects link 0.554 0.756 276
TRS_NAVITIMESHEET_00_login Timesheet navigation user login 0.774 0.907 330
TRS_NAVITIMESHEET_00_logout Timesheet navigation user logout 0.264 0.374 330
TRS_NAVITIMESHEET_00_startup Timesheet navigation session startup 0.059 0.063 25
TRS_NAVITIMESHEET_01_timeperiod_ascending Time period sorting: ascending 0.176 0.253 330
TRS_NAVITIMESHEET_01_timeperiod_descending Time period sorting: descending 0.168 0.233 330
TRS_NAVITIMESHEET_01_timesheetlink Timesheet link 0.178 0.243 330
TRS_NAVITIMESHEET_02_cancel_tmshtproject Timesheet project view cancel 0.231 0.324 330
TRS_NAVITIMESHEET_02_opentimesheet Open a time sheet 0.277 0.374 330
TRS_NAVITIMESHEET_02_opentimesheetproj Time sheet project link 1.11 1.352 330
TRS_NAVPRJ_00_login Project navigation user login 0.667 0.766 20
TRS_NAVPRJ_00_logout Project navigation user logout 0.265 0.344 20
TRS_NAVPRJ_00_startup Project navigation startup 0.056 0.063 20
TRS_NAVPRJ_01_OpenProjList Project List 0.551 0.766 180
TRS_NAVPRJ_02_sort_proj_name_acs Sort by Project name: ascending 0.565 0.776 180
TRS_NAVPRJ_02_sort_proj_name_desc Sort by Project name: descending 0.571 0.766 180
TRS_NAVPRJ_03_sort_proj_id_acs Sort by Project ID: ascending 0.603 0.867 180
TRS_NAVPRJ_03_sort_proj_id_desc Sort by Project ID: descending 0.591 0.816 180
TRS_NAVPRJ_04_open_proj_overview Project navigation user overview 1 1.281 180
TRS_NAVPRJ_04_open_proj_properties Project properties tab 0.972 1.241 180
TRS_NAVRES_00_login Resource navigation user login 0.711 0.877 20
TRS_NAVRES_00_logout Resource navigation user logout 0.23 0.263 20
TRS_NAVRES_00_startup Resource navigation session startup 0.06 0.063 20
Action Description Response
Time Percentile
of Tests Run
TRS_NAVRES_01_firstpage Resource list first page 0.523 0.756 193
TRS_NAVRES_01_lastpage Resource list last page 0.397 0.575 193
TRS_NAVRES_01_nextpage Resource list next page 0.521 0.726 193
TRS_NAVRES_01_resasc Resource list sorting: ascending 0.513 0.695 193
TRS_NAVRES_01_resdesc Resource list sorting: descending 0.517 0.726 193
TRS_NAVRES_01_resfilter Resource list filter for opened 0.676 0.917 193
TRS_NAVRES_01_reslink Resources link 0.484 0.655 193
TRS_NAVRES_02_calendertab Resource calendar tab 0.191 0.273 193
TRS_NAVRES_02_openres Resource open Resource 0.485 0.625 193
TRS_NAVRES_02_projtab Resource project tab 0.47 0.665 193
TRS_NAVRES_02_skillstab Resource skills tab 0.189 0.293 193
Averages 0.4 0.531
Appendix D: Small User Base Test Detailed Results
Below are the detailed Mercury LoadRunner results for the Small User Base test.
Action Description
Average Response
Time
90th Percentile
Number of Tests Run
TRS_ENTERTIME_00_login Time enter user login 0.76 0.867 42
TRS_ENTERTIME_00_logout Time enter user log out 0.305 0.514 42
TRS_ENTERTIME_00_startup Time enter script start session 0.076 0.103 4
TRS_ENTERTIME_01_timeperiod_ascending Sorting time periods ascend 0.212 0.444 42
TRS_ENTERTIME_01_timesheetlink Click time sheet link 0.2 0.514 42
TRS_ENTERTIME_02_opentimesheet Open a time sheet 0.205 0.253 42
TRS_ENTERTIME_02_savetimesheet Save time sheet 0.374 0.404 42
TRS_ENTERTIME_03_submitTS Submit Time Sheet 0.758 0.867 42
TRS_ENTERTIME_04_listsubmittedTS List submitted time sheet 0.197 0.434 42
TRS_ENTERTIME_04_opensubmittedTS Open submitted Time sheet 0.184 0.193 42
TRS_ENTERTIME_04_returnTS Return Time Sheet 0.331 0.454 42
TRS_NAVIGENERAL_00_login General navigation user login 0.719 0.776 15
TRS_NAVIGENERAL_00_logout General navigation user logout 0.258 0.243 15
TRS_NAVIGENERAL_00_startup General navigation session startup 0.061 0.073 15
TRS_NAVIGENERAL_01_AcctSettings Account settings link 0.075 0.103 129
TRS_NAVIGENERAL_01_Organizer Organizer link 0.164 0.203 129
TRS_NAVIGENERAL_01_Overview Overview page 0.31 0.384 129
TRS_NAVIGENERAL_01_ReportsJobs Reports and Jobs link 0.161 0.213 129
TRS_NAVIGENERAL_02_Applications Applications link 0.277 0.354 129
TRS_NAVIGENERAL_02_Assests Assests link 0.321 0.394 129
TRS_NAVIGENERAL_02_Ideas Ideas link 0.2 0.253 129
TRS_NAVIGENERAL_02_OtherInvests Other investments link 0.315 0.414 129
TRS_NAVIGENERAL_02_Portfolios Portfolios link 0.294 0.364 129
TRS_NAVIGENERAL_02_Products Products link 0.276 0.354 129
TRS_NAVIGENERAL_02_Programs Programs link 0.405 0.514 129
TRS_NAVIGENERAL_02_Projects Projects link 0.474 0.595 129
TRS_NAVITIMESHEET_00_login Timesheet navigation user login 0.706 0.806 159
TRS_NAVITIMESHEET_00_logout Timesheet navigation user logout 0.305 0.615 159
TRS_NAVITIMESHEET_00_startup Timesheet navigation session startup 0.077 0.073 13
TRS_NAVITIMESHEET_01_timeperiod_ascending Time period sorting: ascending 0.178 0.253 159
TRS_NAVITIMESHEET_01_timeperiod_descending Time period sorting: descending 0.195 0.364 159
TRS_NAVITIMESHEET_01_timesheetlink Timesheet link 0.169 0.223 159
TRS_NAVITIMESHEET_02_cancel_tmshtproject Timesheet project view cancel 0.229 0.303 159
TRS_NAVITIMESHEET_02_opentimesheet Open a time sheet 0.242 0.303 159
TRS_NAVITIMESHEET_02_opentimesheetproj Time sheet project link 0.996 1.19 159
TRS_NAVPRJ_00_login Project navigation user login 0.669 0.968 9
TRS_NAVPRJ_00_logout Project navigation user logout 0.25 0.534 9
TRS_NAVPRJ_00_startup Project navigation startup 0.053 0.063 9
TRS_NAVPRJ_01_OpenProjList Project List 0.463 0.555 76
TRS_NAVPRJ_02_sort_proj_name_acs Sort by Project name: ascending 0.498 0.565 76
TRS_NAVPRJ_02_sort_proj_name_desc Sort by Project name: descending 0.528 0.655 76
TRS_NAVPRJ_03_sort_proj_id_acs Sort by Project ID: ascending 0.556 0.897 76
TRS_NAVPRJ_03_sort_proj_id_desc Sort by Project ID: descending 0.505 0.635 76
TRS_NAVPRJ_04_open_proj_overview Project navigation user overview 0.911 1.18 76
TRS_NAVPRJ_04_open_proj_properties Project properties tab 0.853 1.038 76
TRS_NAVRES_00_login Resource navigation user login 0.677 0.756 9
TRS_NAVRES_00_logout Resource navigation user logout 0.242 0.414 9
TRS_NAVRES_00_startup Resource navigation session startup 0.055 0.063 9
TRS_NAVRES_01_firstpage Resource list first page 0.458 0.645 83
Action Description
Average Response
Time
90th Percentile
Number of Tests Run
TRS_NAVRES_01_lastpage Resource list last page 0.357 0.454 83
TRS_NAVRES_01_nextpage Resource list next page 0.456 0.625 83
TRS_NAVRES_01_resasc Resource list sorting: ascending 0.44 0.544 83
TRS_NAVRES_01_resdesc Resource list sorting: descending 0.435 0.534 83
TRS_NAVRES_01_resfilter Resource list filter for opened 0.552 0.645 83
TRS_NAVRES_01_reslink Resources link 0.388 0.484 83
TRS_NAVRES_02_calendertab Resource calendar tab 0.17 0.253 83
TRS_NAVRES_02_openres Resource open Resource 0.424 0.514 83
TRS_NAVRES_02_projtab Resource project tab 0.417 0.534 83
TRS_NAVRES_02_skillstab Resource skills tab 0.211 0.404 83
Averages 0.366 0.481
