• No results found

Benchmark Factory for Databases 6.5. User Guide

N/A
N/A
Protected

Academic year: 2021

Share "Benchmark Factory for Databases 6.5. User Guide"

Copied!
345
0
0

Loading.... (view fulltext now)

Full text

(1)
(2)

used or copied only in accordance with the terms of the applicable agreement. No part of this guide may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying and recording for any purpose other than the purchaser’s personal use without the written permission of Quest Software, Inc.

The information in this document is provided in connection with Quest products. No license, express or implied, by estoppel or otherwise, to any intellectual property right is granted by this document or in connection with the sale of Quest products. EXCEPT AS SET FORTH IN QUEST'S TERMS AND CONDITIONS AS SPECIFIED IN THE LICENSE AGREEMENT FOR THIS PRODUCT, QUEST ASSUMES NO LIABILITY WHATSOEVER AND DISCLAIMS ANY EXPRESS, IMPLIED OR STATUTORY WARRANTY RELATING TO ITS PRODUCTS INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTY OF

MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT. IN NO EVENT SHALL QUEST BE LIABLE FOR ANY DIRECT, INDIRECT, CONSEQUENTIAL, PUNITIVE, SPECIAL OR INCIDENTAL DAMAGES (INCLUDING, WITHOUT LIMITATION, DAMAGES FOR LOSS OF PROFITS, BUSINESS INTERRUPTION OR LOSS OF INFORMATION) ARISING OUT OF THE USE OR

INABILITY TO USE THIS DOCUMENT, EVEN IF QUEST HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. Quest makes no representations or warranties with respect to the accuracy or completeness of the contents of this document and reserves the right to make changes to specifications and product descriptions at any time without notice. Quest does not make any commitment to update the information contained in this document.

If you have any questions regarding your potential use of this material, contact: Quest Software World Headquarters

LEGAL Dept 5 Polaris Way

Aliso Viejo, CA 92656 email:[email protected]

Refer to our Web site (www.quest.com) for regional and international office information. Quest, Quest Software, the Quest Software logo, AccessManager, ActiveRoles, Aelita, Akonix, Benchmark Factory, Big Brother, BridgeAccess, BridgeAutoEscalate, BridgeSearch, BridgeTrak, BusinessInsight, ChangeAuditor, CI Discovery, Defender, DeployDirector, Desktop Authority, Directory Analyzer, Directory Troubleshooter, DS Analyzer, DS Expert, Foglight, GPOADmin, Help Desk Authority, Imceda, IntelliProfile, InTrust, Invirtus, iToken, JClass, JProbe, LeccoTech, LiteSpeed, LiveReorg, LogADmin, MessageStats, Monosphere, NBSpool, NetBase, NetControl, Npulse, NetPro, PassGo, PerformaSure, Point, Click, Done!, Quest vToolkit, Quest vWorkSpace, ReportADmin, RestoreADmin, ScriptLogic, SelfServiceADmin, SharePlex, Sitraka, SmartAlarm,

(3)

Spotlight, SQL Navigator, SQL Watch, SQLab, Stat, StealthCollect, Storage Horizon, Tag and Follow, Toad, T.O.A.D., Toad World, vAutomator, vConverter, vEcoShell, VESI,vFoglight, vPackager, vRanger, vSpotlight, vStream, vToad, Vintela, Virtual DBA, VizionCore, Vizioncore vAutomation Suite, Vizioncore vEssentials, Vizioncore vWorkflow, WebDefender, Webthority, Xaffire, and XRT are trademarks and registered trademarks of Quest Software, Inc in the United States of America and other countries. For a complete list of Quest Software’s trademarks, please seehttp://www.quest.com/legal/trademark-information.aspx. Other trademarks and registered trademarks are property of their respective owners.

Benchmark Factory for Databases User Guide

(4)

New For This Release

18

New for Benchmark Factory 6.5 18

New for Benchmark Factory 6.1.1 18

New for Benchmark Factory 6.1 18

New for Benchmark Factory 6.0 18

Understanding Benchmark Factory

19

About Benchmark Factory

19

The Benchmark Factory Environment

19

Integration with Other Quest Software Products 20

Understanding the Benchmark Factory Environment

20

Understanding Benchmark Factory Terminology

21

How Benchmark Factory Works

24

Create a Script With Benchmark Factory

25

Benchmark Factory Components

32

Benchmark Factory Console

32

Agent(s)

33

Run Reports Node

34

Cutting and Pasting of Run Reports 34

Testing Server Throughput 34

Testing SQL Scalability 35

Testing Real Application Clusters 35

(5)

Views

38

Script View 38

Jobs Queue View 39

Properties View 40 Agents View 40 Output View 40 Properties 41

How Do I...

71

Overview

71

How do I...

71

User Scenario Wizard

72

Profile Creation Wizard

72

Overview 72

Edit a Profile

74

Submit Job Dialog

74

Run Reports Node

76

Cutting and Pasting of Run Reports 77

Testing Server Throughput 77

Testing SQL Scalability 78

Testing Real Application Clusters 78

Advanced Creation Objects (Object Mapping)

79

Benchmark Objects Wizard

79

Changing Scaling Factors

81

To Change Scale Factors 82

Execute File Wizard

83

Overview 83

(6)

Script Assist 85

SQL Scalability Wizard

86

Overview 86 OverviewTab 87 Transactions 88 Summary 90 Messages 90

Copy a Load Scenario to a Replay, Mix, Goal, or Scalability Test

91

Running SQL Preview

92

Replace Child Latencies

93

Goal Testing Quick Start

94

Purpose of this Quick Start 95

Actions Performed During This Quick Start 95

Use Global Search and Replace

104

Creating an ODBC Trace File

105

Import Profiles from the Repository

105

Real Application Clustering (RAC) Quick Start

106

Overview 106

Purpose of this Quick Start 106

Actions Performed During This Quick Start 106

Save the Selected Job to the Script View

116

Schedule A Job

116

About Latency

118

Set the Number of Users on an Agent

119

Stagger User Startup

120

(7)

Benchmark Factory Console 121

Benchmark Factory Agent 121

Running a Job With Multiple Agents 121

Creating Load Scenarios

123

Load Scenario Wizard

123

Overview 123

Settings

125

About Settings

125

General Settings

125

Benchmarks Settings

127

Timing Settings

128

User Load (Measurement Intervals)

129

Latency Settings

131

Repository Settings

133

Agent Settings

134

Drivers

136

Driver Selection

136

Adding BFScript to the Profile Page 136

IBM DB2CLI 137

Microsoft SQL Server (Native) 138

Microsoft SQL Server (ODBC) 138

MySQL (ODBC) 139 MySQL ODBC 139 Oracle 139 Sybase 140

Profiles

142

Overview

142

(8)

Import Profiles

143

Profile Properties

143

Overview 143

General Tab 144

Options Tab 145

Profile Properties Real-Time Counters Tab 146

Clustering Tab 149

Wizards

150

Overview

150

Benchmark Objects Wizard

150

Benchmark Objects Wizard 150

Data Repository Migration Wizard

152

Data Repository Migration Wizard 152

Execute File Wizard

153

Execute File Wizard 153

Load Scenario Wizard

154

Load Scenario Wizard 154

Industry Standard Benchmark (TPC-C, TPC-H...) 155

Validate Oracle RAC Cluster Scalability 156

Test Database Scalability 157

Test SQL For Scalability 159

Replay Load from an Oracle Trace File 160

Test Stored Procedures for Scalability 162

Replay Load from SQL Server 2000/2005 Trace Table 163

Import from Foglight Performance Analysis Repository 164

Import from a Delimited Text File 165

(9)

Import from Oracle Dynamic Performance View 168

Import from SQL Files 169

Create a Custom Load Scenario (Scalability, Goal...) 170

Mix Test 170

Replay Test 171

Goal Test 172

Scalability Test 174

Profile Creation Wizard

175

Profile Creation Wizard 175

BFScript

176

BFScript Wizard 176

SQL Scalability Wizard

179

SQL Scalability Wizard 179

Submit Job Dialog

184

Submit Job Dialog 184

Transaction Wizard

186

Transaction Wizard 186

Test Stored Procedures for Scalability 187

Create SQL Statement Transaction 188

Create Prepared SQL Statement Transaction(s) 189

Create Execute Prepared SQL Statement Transaction 193

Import Transactions 193

User Scenario Wizard

194

User Scenario Wizard 194

Import and Replay Load Scenarios From an Oracle Trace File 194

Replay Load from a SQL Server Trace Table 195

Import from a Delimited Text File 196

(10)

Testing Results (Run Reports)

198

Benchmark Factory Testing Results

198

Benchmark Factory Testing Results Terminology

198

Exporting Test Results to Excel Using the Command Line

199

Real-Time Load Testing Results

200

Overview-Viewing Real-time Statistics Workloads 200

Change Graph Views 200

Run Status Node 203

Real-Time Statistics Node 203

Summary Node 205

Transaction Node 206

Benchmark Factory Run Reports

207

Run Reports Node 207

Change Graph Views 210

Run Reports Viewer 212

View Load Testing Results 213

Using Benchmark Factory Run Reports 214

View Multiple Run Results 220

Export Test Results to Excel 221

Benchmarks

222

Overview

222

Realistic Expectations When Using Benchmarks 222

What Benchmarks Measure 222

Provided Benchmarks 222

AS3AP Benchmark

223

Best Practices 223

(11)

How the Scalable Hardware Benchmark Works 224

Scaling Factor 224

Best Practices 224

TPC-B Benchmark

225

Overview 225

Certification of Transaction Processing Council (TPC) Testing Results 225

Best Practices 225

Scaling Factor 226

TPC-C Benchmark

226

Overview 227

TPC-C Tables 227

Certification of Transaction Processing Council (TPC) Testing Results 228

Best Practices 228

TPC-D Benchmark

229

Overview 229

Certification of Transaction Processing Council (TPC) Testing Results 229

Best Practices 229

Scaling Factor 230

TPC-E Benchmark

230

Overview 230

Certification of Transaction Processing Council (TPC) Testing Results 230

Best Practices 231

Scaling Factor 231

TPC-H Benchmark

231

Overview 231

Certification of Transaction Processing Council Testing Results 232

Best Practices 232

(12)

About Scripts

234

Using Scripts in Username and Password Fields

236

Dates and Numbers

236

$BFCreditCardExp 236 $BFCurrentDate 236 BFCurrentDateTime 237 $BFDate 238

File Access

239

$BFFileArray 239

Global Variable

240

$BFGetGlobalVar 240 $BFSetGlobalVar 240 $BFSetGlobalVarRtn 241

Name and Address

241

$BFAddress 241 $BFAddress2 241 $BFCity 242 $BFCompany 242 $BFCountry 243 $BFEmail 243 $BFFirstName 243 $BFFullName 244 $BFLastName 244 $BFMiddleInitial 244 $BFPhone 245 $BFState 245 $BFZipCode 246

(13)

Numerical Manipulation

246

$BFFormat 246 $BFProd 247 $BFSum 247

Random Numbers

248

$BFRand 248 $BFRandRange 248 $BFURandRange 249 $BFURand 249

Random String

250

$BFCreditCard 250 $BFRandList 250 $BFRandMultiList 251 $BFRandStr 252 $BFURandList 253 $BFList 254

String Manipulation

255

$BFAsc 255 $BFChr 255 $BFConcat 255 $BFLeft 256 $BFLen 256 $BFLower 257 $BFMid 257 $BFRight 258 $BFTrim 258 $BFTrimLeft 258 $BFTrimRight 259

(14)

Test Info

260

$BFGetVar 260 $BFMaxNode 260 $BFNode 261 $BFNumberOfIterations 261 $BFProfile 262 $BFRunID 262 $BFSetVar 263 $BFSetVarRtn 263 $BFUserCounter 264 $BFUserID 264 $BFUserLoad 265 $BFNextUserload 265 $BFPrevUserload 265

Quick Starts

267

About the Quickstarts

267

Real Application Clustering (RAC) Quick Start

267

Overview 267

Purpose of this Quick Start 268

Actions Performed During This Quick Start 268

Upgrading Your Database Server Quick Start

278

Overview 278

Purpose of this Quick Start 278

Actions Performed During this Quick Start 279

Oracle Trace Import-Database Replay Quick Start

286

Purpose of this Quick Start 286

(15)

Standard Benchmark Quick Start

293

Purpose of this Quick Start 293

Actions Performed During this Quick Start 293

Goal Testing Quick Start

300

Purpose of this Quick Start 300

Actions Performed During This Quick Start 301

Troubleshooting

310

Agent Connection

310

Use Benchmark Factory with SQL Server 2005 Client

310

MySQL Initialization Settings

310

Appendix

311

Change Graph Views

311

Show Data/Show Graph 311

Graph Legend 312

Toolbar 312

Print 313

Copy Data to Clipboard 313

Copy Graph to Clipboard 313

Load Configuration 313

Save Configuration 313

Set as Default 313

Clear Configuration 313

Customize List Controls

313

Create a SQL 7 Trace Table Using the SQL Server Profiler

314

Create a SQL 2000 Trace Table Using the SQL Server Profiler

315

Create a SQL 2005 Trace Table Using the SQL Server Profiler

316

Oracle Instant Client Installation

317

(16)

Store Procedure Examples

318

Oracle Trace File Activation

320

Support Bundle

321

System Requirements/Upgrade Requirements/Supported Databases

322

System Requirements 322

Shortcut Keys

324

Copyright, Licensing, and Licensing Agreement

327

Licensing and Virtual User Packs

328

Modify the Benchmark Factory License 329

Modify the Benchmark Factory Virtual User Pack 330

330

330

Appendix:Contact Quest

342

Contact Quest/About Quest Software

342

About Quest 342

Contacting Quest Software 342

Contacting Quest Support 342

(17)

Getting Started

Getting Started

Benchmark Factory is an intuitive, easy to use database load testing tool. The following links provide you with information to quickly start the testing process and gather results.

l Video: Take a Tour of Benchmark Factory l About Benchmark Factory (page 19)

l Understanding the Benchmark Factory Environment (page 20) l Understanding Benchmark Factory Terminology (page 21) l How Benchmark Factory Works (page 24)

l Benchmark Factory Components

l Create a Script With Benchmark Factory (page 25) l Benchmark Factory Testing Results (page 198) l Quick Starts

l Real Application Clustering (RAC) Quick Start (page 267) l Upgrading Your Database Server Quick Start (page 278) l Oracle Trace Import-Database Replay Quick Start (page 286) l Standard Benchmark Quick Start (page 293)

l Goal Testing Quick Start (page 300)

l Quest Benchmark Factory Web Site (contains additional videos and white papers)

Benchmark Factory Community

Get the latest product information, find helpful resources, and join a discussion with the Benchmark Factory for Databases team and other community members. Join the Benchmark Factory for Databases community athttp://www.benchmarkfactory.inside.quest.com.

(18)

New For This Release

New for Benchmark Factory 6.5

l New Oracle Trace Record and Playback Wizard-A new wizard has been added that walks users through the entire process of capturing the Oracle workload, including the database backup.

l Enhanced Connection Profile Page-The connection profile pages have been enhanced to be more specific to the particular database vendor (Oracle, Microsoft SQL Server, etc.). l Oracle Profile Improvements-The Oracle profile now supports multiple Oracle Homes,

Direct Connections, as well as being able to specify "Connect As". The Oracle Profile also has a new Statistics page to allow the user to automatically capture Statspack or AWR snapshots during test execution.

l New Activation Window-The new Activation window provides trial users with more information such as expiration date, days remaining, licensing information, etc. l New Licensing Window-A new licensing window is now available to provide better

information about the user's licensing for the different Benchmark Factory licensing keys.

New for Benchmark Factory 6.1.1

This release of Benchmark Factory for Databases provides critical bug fixes.

New for Benchmark Factory 6.1

l Support for 11gR2.

l New global variable support for BFScript. l Enhanced bind types.

New for Benchmark Factory 6.0

l The TPC-E benchmark is now available in this release for both Oracle and Microsoft SQL Server.

(19)

Understanding Benchmark Factory

About Benchmark Factory

Benchmark Factory is a database performance and code scalability testing tool that simulates users and transactions on the database and replays production workload in non-production environments. This enables developers, DBAs, and QA teams to validate that their databases will scale as user load increases, application changes are made, and platform changes are

implemented. Benchmark Factory is available for Oracle, SQL Server, DB2, Sybase, MySQL, and other databases via ODBC connectivity. The following topics provides an overview of the Benchmark Factory Environment:

l The Benchmark Factory Environment (page 19)

l Understanding the Benchmark Factory Environment (page 20) l Understanding Benchmark Factory Terminology (page 21) l How Benchmark Factory Works (page 24)

l Create a Script With Benchmark Factory (page 25)

The Benchmark Factory Environment

Benchmark Factory places enormous stress on a database system, which is typically hard to achieve in a standard testing environment. A system typically breaks under extreme load. By identifying system capacity and performance bottlenecks before they occur, Benchmark Factory facilitates proactive testing, which in turn reduces downtime, development costs, and potential loss of revenue. Benchmark Factory allows you to:

l Determine system throughput and capacity for database systems

l Simulate thousands of concurrent users with a minimal amount of hardware l Find applications that do not scale well with an increase in the number of users l Find breaking points, weak links, or bottlenecks of a system

l Quantify application or server performance

All test results are collected and stored in the repository for data analysis and reporting. Benchmark Factory collects a vast amount of statistics, including overall server throughput (measured in transactions per second, bytes transferred, etc.) and detailed transaction statistics by

(20)

Integration with Other Quest Software Products

Benchmark Factory integrates with otherQuest Softwareproducts to allow IT departments to quickly measure the capacity and performance of their systems to ensure that users will experience fast response times. For instance, Benchmark Factory can run multiple workloads in conjunction with Spotlight™ products. This provides the ability to detect and diagnose issues, allowing users to resolve bottlenecks, slow performance, and application flaws before an application is entered into production.

Understanding the Benchmark Factory Environment

Note: The Benchmark Factory Quick Starts provide procedures that introduce you to the

Benchmark Factory environment. See "About the Quickstarts" (page 267) for more information. The following provides a narrative on the terminology, application, and mechanics of using Benchmark Factory.

Automated Testing, Virtual Users, and Agents

When you load test with Benchmark Factory, you are performing automated testing. Automated testing is the process of using software to test computer hardware or software implying that the software is used instead of actual users. The software acts as a "virtual" user. For example, say you would like to test your database with two hundred virtual users over a given period of time. Benchmark Factory lets you select two hundred virtual users and the length of time you wish to perform the test.

Benchmark Factory usesAgentsto deploy the virtual users. An Agent is a software routine that waits in the background and performs an action when a specified event occurs. One Agent can simulate thousands of virtual users (limited by hardware and workload characteristics) at a time. Each virtual user has their own connection to the system under test.

Understanding Benchmarks

A benchmark is a performance test of hardware and/or software on a system-under-test.

Benchmark Factory provides the option of using industry standard benchmarks during the load testing process. Benchmarks measure system peak performance when performing typical operations.

Benchmark Factory comes equipped with the following industry standard benchmarks: l AS3AP Benchmark (page 223)

(21)

l TPC-C Benchmark (page 226) l TPC-C Benchmark (page 226) l TPC-D Benchmark (page 229) l TPC-E Benchmark (page 230) l TPC-H Benchmark (page 231)

Understanding Benchmark Factory Terminology

The following provides a list of terminology required to understand the Benchmark Factory load testing process.

l A script is the user's workspace that allows you to create and save jobs.

l Scripts are created in the Script View. See "Script View" (page 38) for more information.

l When ready to run, jobs are submitted to the Jobs Queue View. See "Jobs Queue View" (page 39) for more information.

l The Load Scenario Wizard is the starting point for the creation of scripts. See "Load Scenario Wizard" (page 154) for more information.

l A job contains load scenario, execution of a file, creation/deletion of benchmark objects, or a combination of each.

l A load scenario is an assembled test comprised of user scenarios and/or transactions. These tests can be run with multiple virtual users. A load scenario can be one of the following:

l Mixed Workload: A Mixed Workload test runs for a specified time at each predetermined user load level. Each user will run a transaction mix based upon the weights defined on the transactions tab. For example, if a test has two transactions, A and B, with A having a weight of one and B having a weight of four, on average B will run four times for every time A is run once. The run order will be randomly generated for each user so they are not all running the same transaction simultaneously. That run order is used for that user each time the test is performed to ensure reproducible results.

l Goal Test: A goal test is used to find maximum throughput or response time goals. A transaction mix is executed at user load levels, determined by setting a beginning, ending, and interval value. When run, the specified goal criterion is evaluated at the end of each iteration and the test ends once the goal or maximum user load has been reached.

l Replay Test: A Replay Test runs multiple transactions with each one running independently on a specified number of users. The test will run until the defined number of executions for each transaction or a specified time limit is reached.

(22)

test could have two transactions, A and B, and two userloads of 10 and 20, with an iteration length of one minute. Transaction A would execute continually for one minute at userload 10, then B would do the same. Next A will run at userload 20, followed again by test B, for a total time of 4 minutes.

Click here for load scenario creation procedures.

l A user scenario is a series of one or more transactions to be executed in order to make a single transaction, normally associated with user behavior simulated against the system-under-test. An Execute SQL Statement from a file User Scenario will execute SQL statements as defined in an XML file. See "User Scenario Wizard " (page 194) for more information.

l A transaction is a single unit of work in the testing process, used in load scenarios and user scenarios. For example, a SQL statement. Click here for procedures to create a transaction.

l Execute a file allows you to execute a file during the running of a job. See "Execute File Wizard" (page 153) for more information.

l Create or delete benchmark objects configures a system-under-test for industry standard benchmark tests to measure system performance. Each standard benchmark has been developed with specific system configuration requirements, that include tables, indexes, data, etc. See "Benchmark Objects Wizard" (page 150) for more information.

(23)
(24)

How Benchmark Factory Works

The following steps provide an overview of how Benchmark Factory components interact during the load testing process.

1. The Benchmark Factory Console (page 32) implements the workload testing process and controls one or more distributed agent machines. Each agent machine can simulate thousands of users. Each simulated user executes transactions and records statistics. 2. Benchmark Factory Agent(s) (page 33)machines simulate virtual users. The Agents send

transactions to the system-under-test. The Agents record statistics that includes how much data the resulting transaction contained and how long it took to get the results. At the end of an iteration, each agent machine reports its findings back to Benchmark Factory. 3. A server (system-under-test) is the database Benchmark Factory connects to. Benchmark

Factory is server neutral and network protocol independent. Benchmark Factory uses vendor client libraries of the system-under-test. Any system that the client software can support, Benchmark Factory can support. Its only requirement is that Agent machines must be able to connect to a server through an appropriate manner and it supports the test system.

4. The Repository Manager (page 36) stores all testing data. 5. Results are viewed from Run Reports Node (page 207).

(25)

Create a Script With Benchmark Factory

The following provides a narrative discussion on how to create a script using Benchmark Factory. This process consists of following steps:

1. Set up Benchmark Factory 2. Create a Script

3. Create a Profile 4. Create a Job 5. Run the Script 6. Analyze Test Results

Note: The Benchmark Factory Quick Starts provide procedures that introduce you to the

Benchmark Factory environment. See "About the Quickstarts" (page 267) for more information.

1. Set up Benchmark Factory

The Settings window displays default values used when creating new scripts. Changes to these settings affect only new load scenarios, not existing load scenarios.

To Display Benchmark Factory Settings

l Select Edit | Settings. The settings dialog displays. The Settings window provides the following tabs:

l General Settings (page 125) l Benchmarks Settings (page 127) l Timing Settings (page 128) l Latency Settings (page 131) l Repository Settings (page 133) l Agent Settings

(26)

2. Create a Script

Using the Benchmark Factory graphical user interface, you can create scripts in the Script View. You open one script a time when using Benchmark Factory. See "Benchmark Factory Console" (page 32) for more information.

All tasks required to complete a Benchmark Factory script can be accomplished using wizards. The first wizard you will use is the Load Scenario Wizard. The Load Scenario Wizard takes you through the steps required to create a script and complete a database load test. See "Load Scenario Wizard" (page 154) for more information.

(27)

The first step the Load Scenario wizard asks you to perform is to select a load scenario type from the load scenario dialog. Load Scenarios are an assembled load test comprised of:

l User scenarios: A series of one or more transactions to be executed in order normally associated with a behavior to be tested on the system-under-test. These are the components used to build a load scenario.

l Transaction: A single unit of work in the testing process, such as retrieving a Web page, executing a SQL statement, writing a file, or sending an email.

After selecting the desired task type, the Load Scenario Wizard will guide you through the steps for script creation, depending on which type of task you select.

3. Create a Profile

Profiles are sets of information that Benchmark Factory uses to communicate with the system-under-test . This information includes, server name, IP address, and user name. You create profiles using the:

(28)

4. Create a Job

A job holds a load scenario, execution of a file, creation/deletion of benchmark objects, or a combination of each. Jobs are created in Benchmark Factory when a load scenario is added to a script. A script can hold as many jobs as you chose to create.

(29)

Change Script Properties

Benchmark Factory script properties can be changed or adjusted to meet load testing requirement. You change properties by clicking on the node of the property you wish to change. For example, if you wish to change the user load test you click on the Load Scenario node. SeeAbout the Script Nodefor more information.

To change the User Load properties, select the User Load tab and select the desired number of users.

(30)

5. Run the Script

After completing the wizards and setting in the desired properties, you run the job by selecting the Job Node and clicking the Submit Job icon , or right-clicking and selecting Submit Job. See "Submit Job Dialog " (page 184) for more information.

6. Analyze Test Results

Benchmark Factory provides a number of metrics to analyze a system-under-test. These include: l Response, retrieval, and transactions times

l Transaction throughput in transactions per second (TPS) l Amounts of data transferred in bites per second (BPS) l Errors incurred

(31)

l Any user defined counters (Windows NT/2000 Performance Monitor) added using the Benchmark Factory Console.

Benchmark Factory provides the following testing results:

l Real time statistics (See "Real-Time Statistics Node " (page 203) for more information.) l Run Reports (See "Run Reports Node " (page 207) for more information.)

l Testing results exported to Excel (See "Export Test Results to Excel" (page 221) for more information.)

(32)

Benchmark Factory Console

The Benchmark Factory console implements the database workload testing process. This interface is where:

l Scripts are created and initiated. A script is the user workspace that allows you to design and save jobs.

l Jobs are created and submitted to the Jobs Queue View. See "Jobs Queue View" (page 39) for more information.

l All scriptpropertiesare set. SeeAbout the Script Nodefor more information. l Settings can be made. See "About Settings" (page 125) for more information.About

Scripts (page 234)

l Users can access test results. See "Benchmark Factory Testing Results" (page 198) for more information.

l Agents are managed. See "Agent(s)" (page 33) for more information. The Benchmark Factory Console contains the following views:

l Script View (page 38) l Jobs Queue View (page 39) l Properties View (page 40) l Agents View (page 40) l Output View (page 40)

(33)

Agent(s)

Benchmark Factory Agent(s) reside on each physical client machine and spawn multiple virtual-user sessions. Each Agent generates an information screen with tabs that contain a variety of Agent information. Benchmark Factory can control hundreds of Agent machines. Each Agent can simulate up to 1500 virtual users, depending upon hardware and test configuration. Virtual users simulate the load against the system-under-test. Each virtual user is a separate thread, acting independently of the other virtual users, with its own connection to the system-under-test. Each virtual user tracks its own statistics that includes transaction times and the number of times a transaction executes.

The Agent provides the follow tabs: l Agent Test Status Tab l Agent Virtual Users Tab l Agent Transaction Tab l Agent Output Tab

(34)

Run Reports Node

Benchmark Factory Run Reports is a separate executable that opens outside of Benchmark Factory and provides a comprehensive and detailed collection of database load testing results. With Benchmark Factory you can drill down into a database to view a wide array of information and statistics that gives you accurate insight into database performance. Run Reports Viewer allows you to access Benchmark Factory load testing results. See "Run Reports Viewer" (page 212) for more information.

Note: Three instances of Run Reports can be viewed at one time.

Cutting and Pasting of Run Reports

From Benchmark Factory run reports you can cut and paste test results into the Clipboard. See Cut and Pasting Reportsfor more information.

Testing Server Throughput

Server throughput is the measurement of database capacity. Benchmark Factory allows you to view a variety of server throughput statistics that include:

l Transaction/Second (TPS) l Transaction Time l Bytes/Second (BPS) l Rows/Second (RPS) l Total Bytes l Total Errors l Total Rows l Response Time

(35)

Testing SQL Scalability

The Benchmark Factory transaction comparison graph provides users with a comparison of the individual transaction performance as they compare to each other. This is most useful when analyzing a SQL Scalability test.

Testing Real Application Clusters

Benchmark Factory allows you to load test Oracle Real Application Clusters (RAC) to determine system capacity. The cluster performance graph allows users to view the performance of

(36)

Repository Manager

Note: If you create a new Benchmark Factory 5.5 repository, earlier versions of Benchmark

Factory will not work against this repository.

The Repository is a database where all of the test results are stored. Benchmark Factory inserts test results into the repository and provides an easy way to access the data. By default, the Repository is a MySQL database that resides on the same machine as Benchmark Factory for Databases. The Repository can reside on another database server if required. Tto change the database select the Data Source Name of the ODBC connection for the new database. Note: If the database structure does not exist on the selected database, a prompt to create the

structure will appear when OK is clicked.

The supported repository database types are: l Sybase Adaptive Server Anywhere l Microsoft SQL Server

l MySQL l Oracle

To launch the Repository Manager

1. Select Start | All Programs | Quest Software | Benchmark Factory for Databases | Repository | Repository Manager. The Repository Manager dialog displays.

(37)

Section Field/Icon Description Connection

Parameters

Data Source Name

Data Source name of the ODBC connection used to connect to the repository database. ODBC Driver Current ODBC driver

User Name The User Name used to log into the selected database.

Password The Password associated with the user name used to log into the database.

Displays the ODBC connection information dialog for the selected data source.

Displays the ODBC Data Source

Administrator dialog, used to add and edit ODBC connections.

Tests the connection of the currently selected ODBC Data Source.

Maintenance Creates the repository objects on the selected database.

Deletes the repository objects on the selected database.

Warning: This will delete all test results

stored in the Repository.

Launches the Data Migration Wizard. See "Data Repository Migration Wizard" (page 152) for more information.

(38)

Views

Script View

The Script View displays the currently open script and provides user workspace that allows you to design and save jobs. When ready to run, jobs are submitted to the Jobs Queue. The Script View Toolbar provides user functionality when creating or editing load testing scripts. Note: You must be in the script view to create jobs and edit jobs.

Script View Toolbar

The Script View toolbar provides user functionality when creating or editing load testing scripts.

Creates a blank job. You must click on the Script Node to active the icon.

Displays the Load Scenario Wizard (page 154)

(39)

Displays the Transaction Wizard (page 186)

Displays the Benchmark Objects Wizard (page 150)

Displays the Execute File Wizard (page 153)

Displays the Submit Job Dialog (page 184)

Moves an item up in the Script View

Moves an item down in the Script View

Jobs Queue View

The jobs view displays the currently open jobs. From the Jobs Queue, jobs are run, status can be viewed, and real-time testing results can be viewed. The Jobs toolbar provides user functionality when running a job.

(40)

Jobs Queue icons provides user functionality when running jobs.

Deletes the selected job from the Jobs Queue.

Saves the selected job to the Script View. See "Save the Selected Job to the Script View" (page 116) for more information.

Expands all jobs when the root is selected. Collapses all jobs when the root is selected.

Properties View

The Properties View allows you to monitor and managed scripts. Properties can be viewed for the:

l Script Node Properties (page 41) l Jobs Queue Node Properties (page 55)

Agents View

The Agent view displays the status of connected agents.

Output View

The Output window displays errors and messages that occur outside of running a job. For example, a .dll error or re-registering a benchmark.

(41)

Properties

About the Properties View

Benchmark Factory properties allow you to: l Adjust script parameters

l View script properties

l Access Benchmark Factory Run Reports See "Run Reports Node " (page 207) for more information.

l Viewrealtime statistics (See "Real-Time Statistics Node " (page 203) for more information.

l Set and view latency See "About Latency " (page 118) for more information. The following provides Benchmark Factory properties definitions:

l Script (SeeAbout the Script Nodefor more information.)

l Job (See "Jobs Queue Node Properties" (page 55) for more information.) l Latency (See "About Latency " (page 118) for more information.)

l Benchmark Properties See "Benchmark Properties" (page 67) for more information.)

Script Node Properties

Script Node

(42)

Jobs Node

Note: In the Jobs Queue view, not all properties are editable. The Jobs node provides the following tabs:

l General l Job Schedule l Jobs Alerts

l Real-Time Counters

l Global Variables Node (page 56)

Load Scenario Node

Note: In the Jobs Queue view, no properties are editable. The Load Scenario node provides the following tabs:

l Transaction Tab l Goal Test (page 172)

(43)

l Timing Tab

l Options Tab-Mixed Workload, Goal, and Scalability Test (page 48)

User Scenario Node

The User Scenario node properties allows you to: l Delete transactions

l Adjust transaction order

l Adjust latency (See "About Latency " (page 118) for more information.)

User Scenario From File

The User Scenario From File node properties allows you to: l Executing a SQL Statements from a File (page 44)

(44)

When large Oracle trace files are imported into Benchmark Factory, they are converted to editable XML files.

To Edit SQL Statements from a file

1. Click Edit.

2. Click File | Open.

The Open dialog displays.

3. Navigate to and select the required file XML file. 4. Click Open.

(45)

5. Under Session Details, click on the SQL statement you wish to edit. That statement displays in the SQL Statement view.

6. From the Prepare SQL tab, you can click in the upper right-hand corner to: l Run SQL Preview See "Running SQL Preview" (page 92) for more

information.

l Launch the BFScript Wizard See "BFScript Wizard" (page 176) for more information.

(46)

Transaction Node

Note: In the Jobs Queue view, no properties are editable.

The transaction node displays properties based on transaction type.

 To view the Transaction node

(47)

Options Tab-Replay Test

The Repository Properties tab controls what information is saved in the Repository during the running of a test, and allows you to set error handling properties. The database where all Benchmark Factory information is saved. It contains information about the transactions stored in each benchmark DLL and is where all of the statistical information is stored about benchmark executions.

To change Repository properties

1. Click the Load Scenario node. 2. Click the Options tab.

Setting Description

Repository Settings

Save Real-Time Counter Information

Saves Benchmark Factory real time counters and performance monitor counters to the repository during test.

Save User Statistics Saves user level statistics.

Save Distribution Saves transactions per second and response time distribution.

(48)

Options Tab-Mixed Workload, Goal, and Scalability Test

The Options tab specifies Repository, error handling, file execution, and database checkpoint settings.

Section Field/Icon Description

Repository Settings SaveReal-Time Counter Information

Note: To change real-time counters please refer to Profile Properties Real-Time Counter Tab.

Saves counters and performance monitor counters to the Repository during test. Sample rate is determined in job settings.

Save User Statistics Saves user level statistics

(49)

Section Field/Icon Description

second and response time per user distribution. Log Transaction Results to disk Logs the transaction results

of a job to disk.

Note: Checking this option imposes performance penalties.

Error Handling Stop test after first error When checked, the test is stopped when an error is reported.

Execute the following program at the beginning of each iteration

File Name

Note: This field accepts BFScripts.

Name of the program to execute at the beginning of an iteration.

Enforce Timeout Enforces a timeout on the file being executed.

Execute the following program at the end of each iteration

File Name

Note: This field accepts BFScripts.

Name of the program to execute at the end of an iteration.

Enforce Timeout Enforces a timeout on the file being executed so that if the program does not complete in the specified time, it will be stopped, and the load scenario will continue.

Database Checkpoints Perform checkpoint at start of each test iteration

Initiates a database

checkpoint at the beginning of a test iteration.

Perform checkpoints during each iteration

Initiates a database checkpoint during a test iteration.

Number of checkpoints Specifies the number of checkpoints to initiate.

(50)

Run Reports Node

Benchmark Factory Run Reports is a separate executable that opens outside of Benchmark Factory and provides a comprehensive and detailed collection of database load testing results. With Benchmark Factory you can drill down into a database to view a wide array of information and statistics that gives you accurate insight into database performance. Run Reports Viewer allows you to access Benchmark Factory load testing results. See "Run Reports Viewer" (page 212) for more information.

Note: Three instances of Run Reports can be viewed at one time. Cutting and Pasting of Run Reports

From Benchmark Factory run reports you can cut and paste test results into the Clipboard. See Cut and Pasting Reportsfor more information.

Testing Server Throughput

Server throughput is the measurement of database capacity. Benchmark Factory allows you to view a variety of server throughput statistics that include:

l Transaction/Second (TPS) l Transaction Time l Bytes/Second (BPS) l Rows/Second (RPS) l Total Bytes l Total Errors l Total Rows l Response Time

(51)

Testing SQL Scalability

The Benchmark Factory transaction comparison graph provides users with a comparison of the individual transaction performance as they compare to each other. This is most useful when analyzing a SQL Scalability test.

Testing Real Application Clusters

Benchmark Factory allows you to load test Oracle Real Application Clusters (RAC) to determine system capacity. The cluster performance graph allows users to view the performance of

(52)

SQL Statement Node

The SQL statement node displays individual SQL statements and allows you edit a SQL statement or use the BFScript Wizard. See "BFScript Wizard" (page 176) for more information.

Create/Delete Benchmark Objects Node Note: In the Jobs Queue view, no properties are editable.

The Create/Delete Benchmark node allows you to either create the benchmark objects or delete the benchmark objects. Benchmark objects are used by standard benchmark load scenarios. Benchmarks can be added to a job using the Benchmark Objects Wizard. See "Benchmark Objects Wizard" (page 150) for more information.

(53)

Field Description

Benchmark Name of the benchmark whose benchmark properties are being edited.

Scale The factor used to increase the amount of data added to the database.

Valid benchmark scales are dependent upon the benchmark selected. For example, the AS3AP scales by a factor of 10 with valid scales of "1, 10, 20, 30..." and the TPC-C benchmark scales by a factory of 1, with valid scales of "1, 2, 3, 4..."

Displays the Benchmark Properties page and allows you to edit the benchmark scale and map database tables and indexes. See

"Benchmark Properties" (page 67) for more information.

Always delete objects before create

Checking this box deletes benchmark objects before creating new ones.

When unchecked, Benchmark Factory checks for the benchmark tables. If they exist, then the creation will be skipped.

(54)

Note: The scale factor is not checked.

Execute File Node

Note: In the Jobs Queue view, no properties are editable.

The Execute File node allows you to place a file execution in a job as a sibling to a load scenario. The file executes during the running of a job.

Field Description

Execute File Type in or Select the File Name to execute:

Name of the file you wish to execute.

Note: This field accepts BFScripts. See "About Scripts" (page

234) for more information.

Click to open a file.

Enforce Timeout Places a time constraint on the number of seconds a file has to execute.

(55)

Field Description between 30 and 10000

[n] Secs.

Jobs Queue Node Properties About the Jobs Node

Some of the nodes in the Jobs Queue are the same as the Script view.

Note: Not all nodes in the Jobs Queue are editable and provide property viewing only. The following nodes display:

l Jobs Queue Node

l Load Scenario Node (page 56) l User Scenario Node (page 56) l Transaction Node (page 57)

l Create/Delete Benchmark Objects Node (page 57) l Run Reports Node (page 207)

l Run Status Node (page 203)

l Real-Time Statistics Node (page 203) l Summary Node (page 205)

l Messages Node (page 64)

Jobs Node

Note: In the Jobs Queue view, not all properties are editable. The Jobs node provides the following tabs:

l General l Job Schedule l Jobs Alerts

(56)

Global Variables Node

From the global variable node, you can add or delete the required variables.

To add a global variable:

1. Right-click to display the global variable dialog. 2. Enter the name and value.

3. Click Add.

Load Scenario Node

Note: In the Jobs Queue view, no properties are editable. The Load Scenario node provides the following tabs:

l Transaction Tab l Goal Test (page 172)

l User Load Tab (Measurement Intervals) l Timing Tab

l Options Tab-Mixed Workload, Goal, and Scalability Test (page 48)

User Scenario Node

The User Scenario node properties allows you to: l Delete transactions

l Adjust transaction order

(57)

Transaction Node

Note: In the Jobs Queue view, no properties are editable.

The transaction node displays properties based on transaction type.

 To view the Transaction node

» Click the Transactions node in the Jobs Queue or Script View.

Create/Delete Benchmark Objects Node Note: In the Jobs Queue view, no properties are editable.

The Create/Delete Benchmark node allows you to either create the benchmark objects or delete the benchmark objects. Benchmark objects are used by standard benchmark load scenarios.

(58)

Field Description

Benchmark Name of the benchmark whose benchmark properties are being edited.

Scale The factor used to increase the amount of data added to the database.

Valid benchmark scales are dependent upon the benchmark selected. For example, the AS3AP scales by a factor of 10 with valid scales of "1, 10, 20, 30..." and the TPC-C benchmark scales by a factory of 1, with valid scales of "1, 2, 3, 4..."

Displays the Benchmark Properties page and allows you to edit the benchmark scale and map database tables and indexes. See

"Benchmark Properties" (page 67) for more information.

Always delete objects before create

Checking this box deletes benchmark objects before creating new ones.

(59)

Field Description

When unchecked, Benchmark Factory checks for the benchmark tables. If they exist, then the creation will be skipped.

Note: The scale factor is not checked.

Run Reports Node

Benchmark Factory Run Reports is a separate executable that opens outside of Benchmark Factory and provides a comprehensive and detailed collection of database load testing results. With Benchmark Factory you can drill down into a database to view a wide array of information and statistics that gives you accurate insight into database performance. Run Reports Viewer allows you to access Benchmark Factory load testing results. See "Run Reports Viewer" (page 212) for more information.

Note: Three instances of Run Reports can be viewed at one time. Cutting and Pasting of Run Reports

From Benchmark Factory run reports you can cut and paste test results into the Clipboard. See Cut and Pasting Reportsfor more information.

Testing Server Throughput

Server throughput is the measurement of database capacity. Benchmark Factory allows you to view a variety of server throughput statistics that include:

l Transaction/Second (TPS) l Transaction Time l Bytes/Second (BPS) l Rows/Second (RPS) l Total Bytes l Total Errors l Total Rows l Response Time

(60)

The Benchmark Factory transaction comparison graph provides users with a comparison of the individual transaction performance as they compare to each other. This is most useful when analyzing a SQL Scalability test.

Testing Real Application Clusters

Benchmark Factory allows you to load test Oracle Real Application Clusters (RAC) to determine system capacity. The cluster performance graph allows users to view the performance of

(61)

Run Status Node

Run Status Node properties displays information on the job when it is running. You can monitor the status of the test, which userload its running, transactions per second (TPS), and a variety of other datapoints that give you insight into the actual performance your database provides under the userload being tested.

Iteration Overruns

Iteration overruns occur at the end of an iteration to allow time for all transactions submitted within the test iteration cycle to complete, so that all transaction statistics can be collected. For example, an agent may execute a transaction during the last five seconds of test iteration, if this transaction takes 15 seconds to complete, an iteration overrun of 10 seconds will occur.

To view the Run Status node

1. Click the Run Status node in the Jobs Queue.

Real-Time Statistics Node

The Real Time Statistics node provides you with real-time graphs and raw data. This data allows you to spot system-under-test issues that may be affecting server performance. The Real Time Statistics node displays real-time counter data. Right-clicking inside the graph displays a drop-down that allows you to change graph settings and view.

To view the Real-Time Statistic node

(62)

2. Select Click here to Add Datapoints. The Add Datapoints dialog displays.

3. Select the desired datapoints to view. 4. Click OK.

(63)

5. To change graph views, right-click. See "Change Graph Views" (page 311) for more information.

Summary Node

The Summary node displays user selected statistics for the job timing period following completion of an iteration.

To view the Summary node

1. Click the Summary node in the Jobs Queue.

(64)

information.

5. Select the desired option.

Messages Node

The Messages node displays: l Job status

l Any errors or warnings

Latency

About Latency

Latency is the speed in which transactions are sent to a server or the period between receiving results of one transaction and sending the next transaction to a serve. Benchmark Factory allows you to select the following latency models and distribution models:

Latency Types Distribution Models

No Delay (page 65) Absolute Distribution (page 66)

Interarrival Time (page 65)

Uniform Distribution (page 66)

Keying Time (page 65) Negative Exponential Distribution (page 66)

Think Time (page 65)  Normal Distribution (page 67)

(65)

No Delay

No Delay means that transactions execute as fast as possible. As soon as one transaction is processed, the next transaction is issued against the server. In the case of a mixed workload test, each virtual user issues transactions as fast as possible. The No Delay option is used when the goal of the test is to stress the system to its limits, without concern for accurately simulating users. With No Delay specified, a relatively low number of users can stress the system to its limits. As such, there is no easy way to correlate N virtual users running with no delay to some number of real users.

Interarrival Time

Interarrival Time derives its name from achieving a specific transaction rate as seen by a server (the rate at which transactions are arriving at a server.) When specifying Interarrival Time, Benchmark Factory is instructed, that regardless of how long a transaction actually takes to execute, to ensure that the transactions arrive at a server at a specific interval.

Keying Time

Keying Time is used to simulate an amount of time performing data entry, before executing a transaction (entering information). In many cases, Keying Time is used with Think Time to provide a delay before and after a transaction execute. Selecting Keying Time inserts a fixed or variable delay before executing a transaction.

Think Time

Think Time is used to simulate an amount of time "to think" about the results of a previous transaction. This could be time spent performing analysis on the results of a database query. Selecting Think Time inserts a fixed or variable delay after executing a transaction.

(66)

About

Benchmark Factory provides the following distribution models l Absolute Distribution (page 66)

l Uniform Distribution (page 66)

l Negative Exponential Distribution (page 66) l  Normal Distribution (page 67)

l  Poisson Distribution (page 67)

Absolute Distribution

Absolute is a fixed length delay. If a 2000 ms delay is specified for Keying Time (page 65) and a 3000 ms delay is specified forThink Time (page 65),each time the transaction should execute, Benchmark Factory waits 2000 ms, then fires the transaction, then waits an additional 3000 ms before deciding which transaction to execute next. If an Inter-Arrival model is chosen with a delay type of Absolute set to 2000 ms, Benchmark Factory marks the time, executes the transaction, and assuming the transaction finishes in less than two seconds, waits until two seconds from the marked time has elapsed before determining which transaction to execute next.

Uniform Distribution

Selecting a Uniform delay instructs that random delay should be chosen, with an equal

probability of being the minimum value, the maximum value, or any value in between. Uniform delays are chosen when it is suspected that the delay is highly random within a range or a minimal amount of statistical analysis has been performed to determine how the actual users react. Suppose a uniform Keying Time (page 65) is selected with a minimum value of 1000 ms and a maximum value of 1500 ms. If the transaction is executed more than 500 times, there is a high probability that each possible delay has been selected at least once. With the other delay types, this is not the case. If the same 2000 ms to 2500 ms uniform delay is set for Interarrival time, the tester essentially is setting the test so that a server sees the transaction every 2 to 2.5 seconds, instead of exactly two seconds as in the Absolute delay time.

Negative Exponential Distribution

Similar to the Normal distribution, Negative Exponential inserts a random delay based on a mathematical model. A Negative Exponential distribution should be chosen when most users

(67)

have latency of the mean latency or less. Negative Exponential is weighted heavily toward smaller latencies being selected relative to the mean and maximum latency.

 Normal Distribution

Normal distributions differ from Uniform delays in that most of the delays chosen by Benchmark Factory will be close to the average, but can vary by as much as ±10% of the mean. While a Uniform delay is used when users have latencies within equal likelihood of being anywhere between two values, Normal distributions are chosen when all users fall within a range, but most of the modeled users have a latency close to the average latency.

 Poisson Distribution

A Poisson distribution is very similar to the Normal distribution and can be used most places where a Normal distribution delay could be used. The biggest difference between a Normal distribution and a Poisson distribution is that Poisson selects discreet values.

Benchmark Properties Benchmark Properties

Benchmark scale factors are used to increase the size of the database testing tables during the creation of benchmark objects, and to correctly execute the benchmark transactions as needed to fully utilize the database objects created.

The Benchmark Properties page displays information on: l Scales (page 68)Scales (page 68)

l Advanced (page 69) l History Tables (page 69)

(68)

The scale dialog specifies the benchmark scale used to create table data.

Field Description

Benchmark Scale The factor used to increase the amount of data added to the database. Valid benchmark scales are dependent upon the benchmark selected. For example, the AS3AP benchmark scales by a factor of 10 with valid scales of "1, 10, 20, 30..." and the TPC-C benchmark scales by a factory of 1, with valid scales of "1, 2, 3, 4..."

Show Empty Tables

Displays any tables that will be created but not populated with data.

Estimated Size The approximate size that the tables will use in the database without indexes. Additional space is needed for indexes.

(69)

Advanced

Advancer mapping provides views of the tables and indexes that allows you to determine where there tables and indexes will be stored in the database. This allows your database to run more efficiently. See "Advanced Creation Objects (Object Mapping)" (page 79) for more information.

History Tables

History tables allows you to set the number of history tables to create. The TPC-B benchmark is made up of only one transaction that updates three tables and inserts a record into a history table. Inserting one record into one history table limits testing performance. The Benchmark Factory properties page allows the user to set the number of history tables to create during a test. The best ratio of history tables to virtual users is based on database configuration and hardware. The number of history tables to use is determined by the tester.

(70)

Field Description

Number of History tables

(71)

How Do I...

Overview

The How Do I section points you to the procedures required to perform load testing.

How do I...

l Create a Load Scenario- This wizard walks you through the steps tocreate and run scriptsand load scenarios.

l Create a User Scenario - This wizard walks you through the steps to create a user scenario.

l Create a Profile- This wizard walks you through the steps to hook up to the data base you are testing.

l Edit a Profile

l Submit a Job- This wizard walks you through the steps to submit a job to theJobs Queuefor testing.

l Use Run Reports

l Use Advanced Creation Objects

l Create Benchmark Objects for Load TestingThis wizard walks you through the steps to create benchmark objects for load testing.

l Execute a File- This wizard walks you through the steps to execute a file outside of Benchmark Factory.

l Change Scale Factors

l Use Benchmark Factory Scripting Capabilities- This wizard walks you through the steps to use Benchmark Factory built-in functions for use in scripts and the load testing process. l Use SQL Scalability Wizard

l Copy a Load Scenario to a Replay Test, Mix Test, or Goal Test l Run SQL Preview

l Replace Child Latencies l Perform a Goal Test

l Use Global Search and Replace l Create a Customized ODBC Trace File l Import Profiles from the Repository

(72)

l Set the Number of Users on an Agent l Stagger User Startup

l Run Benchmark Factory with Multiple Agents

User Scenario Wizard

The User Scenario Wizard allows you to build user scenarios. A user scenario is a series of one or more transactions to be executed in order, normally associated with user behavior simulated against the system-under-test. From the User Scenario Wizard you can:

l Import and Replay Load Scenarios From an Oracle Trace File l Replay load from SQL Server trace table

l Import from a delimited text file l Replay from an ODBC trace File l Create a custom user scenario

Profile Creation Wizard

Overview

Profiles are used by Benchmark Factory to connect to the system-under-test. The profile contains information required to connect to the system-under-test, such as server name, IP address, and user name. The Profile Creation Wizard walks you through steps required to create a Benchmark Factory profile.

To create a profile

1. Select Wizards | Profile Creation. 2. Click Next.

3. From the Driver drop-down list, select the required driver type. 4. Click Next.

(73)

6. Enter the Password.

7. From the TNS Driver drop-down, select the required database. or

Click the Direct tab and enter the required fields.

8. From the Connect as drop, select the required connection, Normal, SYSDBA, or SYSOPER.

9. Select the installed client if applicable.

10. Click the Verify connection before proceeding check box to verify the connection to the specified database.

11. Click Next.

12. Enter a profile name. 13. Click Finish.

(74)

Edit a Profile

To edit a profile

1.

Click the Edit Profile icon .

2. Double-click on a profile to edit or click View Properties in the upper right-hand corner of the dialog.

3. The Edit Profiles dialog displays. The following options are provided. l Adds a profile l Imports a profile l Deletes a profile l

Displays profile properties allowing you to edit the profile

Submit Job Dialog

A job contains load scenario, execute file, creation/deletion of benchmark objects, or a

combination of each. The Submit Job Wizard walks you through the process of adding a job to the jobs queue. See "Jobs Queue View" (page 39) for more information.

To submit a job to the Jobs Queue

1. Select the Job node. 2. Right-click.

(75)

The Submit Job dialog allows you to tailor your job needs and provides the following options:

Section Description

Profile Allows you to select profiles already created.

Displays the Profile Creation Wizard. See "Profile Creation Wizard" (page 175) for more information.

Displays the Profile setting dialog. See "General Tab" (page 144) for more information.

Actions Allows you to:

Run Job: Submits the job to the queue and runs the job.

Run Job when queue becomes idle: Submits the job to the jobs queue and puts it in a ready state so that it will run when the currently ready or running job completes.

(76)

Run Job per schedule: Submits the next job scheduled.

Start Agent and Run Job: Starts the agent and runs the job.

Start Agent and Run Job when queue becomes idle: Starts the agent and runs the job when all previous jobs have finished.

Start Agent and Run Job per schedule: Starts the agent and runs the next job scheduled.

No action: Closes the dialog after you select Submit.

Comments Allows you to add a comment to the job.

Alerts Allows you to send an email, pager, or net-send operator alerts when the job completes.

Job Counters Allows you to addJob Countersto the job.

Stop job after first error

When checked, the job will stop if an error is encountered.

4. After filling in the desired parameters, click Submit.

Run Reports Node

Benchmark Factory Run Reports is a separate executable that opens outside of Benchmark Factory and provides a comprehensive and detailed collection of database load testing results. With Benchmark Factory you can drill down into a database to view a wide array of information and statistics that gives you accurate insight into database performance. Run Reports Viewer allows you to access Benchmark Factory load testing results. See "Run Reports Viewer" (page 212) for more information.

References

Related documents

The Customer's electric service equipment and its installation shall conform to the requirements of the latest edition of the National Electrical Safety Code, National Electrical

Creates a healthy competition Creates a healthy competition between Teams to excel each. between Teams to excel each Stemmed out

We recommended that for business interruption cover whether it is packaged within your policy or deliberately selected that you discuss the amounts with your insurance provider

If you receive this error, please check that the start date entered is within the period of at least one of your professional jobs. If it does, your details may not have been

For convenience, there are many ambiguous sentences in CCAR-145, like “A-check or equivalent…”. The definition about it is actually not clear. To solve the above-mentioned problems,

Are you are my blank page, saying you is says to the sayings in the most orders will not stop you used. 300 I render You

• Note that if you use the Add Existing Item option to add a source file, then Visual Studio doesn’t actually copy the file to the project.. To copy the file to a project, copy the

You are hereby summoned and required to file with the clerk of the harbor court opinion WRITTEN back to a Complaint which please be filed within TEN 10 DAYS.. You cannot be given