PASS
Spirent Journal of Cloud
Application and Security
Services PASS Test
Methodologies
June 2011 Edition
1
Introduction
Today’s Devices Under Test (DUT) represent complex, multi-protocol network elements with an emphasis on Quality of Service (QoS) and Quality of Experience (QoE) that scale to terabits of bandwidth across the switch fabric. The Spirent Catalogue of Test Methodologies represents an element of the Spirent test ecosystem that helps answer the most critical Performance, Availability, Security and Scale Tests (PASS) test cases. The Spirent Test ecosystem and Spirent Catalogue of Test Methodologies are intended to help development engineers and product verification engineers to rapidly develop and test complex test scenarios.
How to use this Journal
This provides test engineers with a battery of test cases for the Spirent Test Ecosystem. The journal is divided into sections by technology. Each test case has a unique Test Case ID (Ex. TC_MBH_001) that is universally unique across the ecosystem.
Tester Requirements
To determine the true capabilities and limitations of a DUT, the tests in this journal require a test tool that can measure router performance under realistic Internet conditions. It must be able to simultaneously generate wire-speed traffic, emulate the requisite protocols, and make real-time comparative
performance measurements. High port density for cost-effective performance and stress testing is important to fully load switching fabrics and determine device and network scalability limits.
In addition to these features, some tests require more advanced capabilities, such as
Integrated traffic, routing, and MPLS protocols (e.g., BGP, OSPF, IS-IS, RSVP-TE, LDP/CR-LDP) to advertise route topologies for large simulated networks with LSP tunnels while simultaneously sending traffic over those tunnels. Further, the tester should emulate the interrelationships between protocol s through a topology.
Emulation of service protocols (e.g., IGMPv3, PIM-SM, MP-iBGP) with diminution. Correct single-pass testing with measurement of 41+ metrics per pass of a packet. Tunneling protocol emulation (L2TP) and protocol stacking.
True stateful layer 2-7 traffic.
Ability to over-subscribe traffic dynamically and observe the effects.
Finally, the tester should provide conformance test suites for ensuring protocol conformance and interoperability, and automated applications for rapidly executing the test cases in this journal.
Further Resources
Table of Contents
Testing Cloud Application and Security Services ...3
CAS_001
Determine IP stateful traffic throughput per RFC 3511 ... 4
CAS_002
Stateful traffic concurrent TCP connection capacity per RFC 3511 ... 7
CAS_003
Determine the maximum TCP connection establishment rate ... 11
CAS_004
Determine the maximum TCP connection tear down rate ... 15
CAS_005
Measure the BBT external to external basic throughput ... 19
CAS_006
Measure the BBT external to internal (virtual) max throughput ... 22
CAS_007
Measure the BBT internal (virtual) to internal (virtual) maximum throughput. 25
CAS_008
Measure performance of HTTP over VPLS over BGP in the cloud ... 28
CAS_009
BBT – External to internal (virtual): scalability and application availability ... 31
CAS_010
BBT – Internal (virtual) to internal (virtual): scalability and application
availability in a virtual environment ... 34
CAS_011
BBT – Internal (virtual) to internal (virtual): security – attacking the virtual
environment ... 37
CAS_012
BBT – External to internal (virtual): security – combined traffic and attacks ... 40
CAS_013
Determine the effects of a denial of service attack on firewall performance per
RFC 3511 Section 5.5 ... 43
CAS_014
Determine HTTP transfer rate of the firewall performance per RFC3511 Section
5.6 ... 48
CAS_015
Measure HTTP transaction rate of the firewall performance per RFC3511
Section 5.7... 52
CAS_016
Determine the capacity of NAT444 in the cloud ... 56
CAS_017
Evaluate DNS A and AAAA record performance under attack ... 60
Appendix A – Telecommunications Definitions ... 72
Appendix B – Stateful Playlist by QoS ... 79
3
Testing Cloud Application and Security
Services
Cloud computing is a general term for anything that involves delivering hosted services over the Internet. These services are broadly divided into three categories: Infrastructure-as-a-Service (IaaS), Platform-as-a-Service (PaaS) and Software-as-a-Platform-as-a-Service (SaaS).
A cloud service has three distinct characteristics that differentiate it from traditional hosting. It is sold on demand, typically by the minute or the hour. It is elastic—a user can have as much or as little of a service as they want at any given time. The service is fully managed by the provider. The consumer needs nothing but a personal computer and Internet access. Significant innovations in virtualization and distributed computing, improved access to high-speed Internet, and financial pressure have accelerated interest in cloud computing.
IaaS provides virtual server instances with unique IP addresses and blocks of storage on demand. Customers use the provider's application program interface (API) to start, stop, access and configure virtual servers and storage. In the enterprise, cloud computing allows a company to pay for only as much capacity as is needed and bring more online as soon as required. Because this pay-for-what-you-use model resembles the way electricity, fuel and water are consumed, it's sometimes referred to as utility computing.
PaaS in the cloud is defined as a set of software and product development tools hosted on the provider's infrastructure. Developers create applications on the provider's platform over the Internet. PaaS providers may use APIs, website portals or gateway software installed on the customer's computer. Currently there are no standards for interoperability or data portability in the cloud. Some providers will not allow software created by their customers to be moved off the provider's platform.
CAS_001 Determine IP stateful traffic throughput per RFC 3511
Abstract
This test determines the throughput of network-layer data traversing the firewall (DUT), as defined in RFC 1242. This test determines the maximum throughput for the DUT per fixed packet size. Suggested packet sizes are 64, 128, 256, 512, 768, 1024, 1218, 1514. Packets should be UDP packets. Each test should run for at least 10 minutes. References: RFC 3511, RFC 2889, RFC 1242. For product verification and engineering.
Description
The tester offers unicast IP packets to the DUT at a constant rate. The test may consist of either bi-directional or unidirectional traffic. For example, an emulated client may offer a unicast stream of packets to an emulated server, or the tester may simulate a client/server exchange by offering bidirectional traffic.
This test employs an iterative search algorithm. For iteration the tester varies the intended load until the maximum rate, at which no packet loss occurs, is found. Since backpressure
mechanisms may be employed, resulting in a difference between the intended load and offered load, the test should be performed in either a packet-based or time-based manner. As with RFC 1242, the term packet is used in place of frame.
Relevance
For the networks that a firewall secures, often the value of the data and systems that they must help protect cost many times that of the firewall and the testing. Deploying a security
infrastructure without understanding its performance and security runs the risk of doing little to protect the network, especially if it also introduces apathy from a false sense of security. Among the reasons that show the criticality of testing firewalls:
Security at load: Often, security flaws do not appear until the network encounters a large load. Attacks can hide more easily within large amounts of traffic, potentially causing problems right when network downtime is most harmful.
Firewall behavior at load and at failure: Firewalls often exhibit different behaviors as they encounter increasing loads. Eventually, with enough traffic, the firewall will fail, providing valuable insights. First, the conditions and behavior leading up to failure are now known, giving the security administrator things to look for in production and providing a useful pre-failure warning. Second, the failure state of the firewall is known—firewalls that fail closed will stop all traffic from passing (essentially a successful denial of service, or DoS, attack), and firewalls that fail open permit all traffic, which is a security failure.
5
Test Category
Testing Stateful Multiplay Benchmark Standards.
PASS
[x] Performance [ ] Availability [ ] Security [ ] Scale
Required Tester Capabilities
The test equipment must have the ability to generate simulated users. Each simulated user must have the ability to schedule full web pages with the ability to schedule specific HTTP objects within TCP tunnels in a predictable order, matched with a predictable source IP and MAC address source.
Topology
Test Procedure
1. Reserve two test ports.
2. Connect tester port 1 as the client side to the configured firewall-capable DUT. Connect port 2 as the server side to the other side of the DUT. Establish the link.
3. Begin Step 1 – Configure the bandwidth performance test.
a. Enter the management IP address of the equipment that is used for this test for the client port.
b. Enter the management IP address of the equipment that is used for this test for the server port.
c. Select the performance test for specific equipment that is used in this test. 4. Begin Step 2 – Test the effect of load, packet size, and duration.
a. Client side:
i. HTTP version 1.1. ii. HTTP GET Per SimUser.
iii. Maximum connections per server 1.
iv. Maximum transaction per connection 10 or 50. v. User think time 30 seconds.
ii. Maximum requests per connection 64 or 10 (if client is set to more than 10). iii. Server type Irrelevant (only changes HTTP headers).
iv. Packet size 64b, 128b, 256b, 512b, 768b, 1024b, 1218b, 1514b. 5. End.
Variables & Relevance
Variable Relevance
Packet sizes – 64b, 128b, 256b, 512b, 768b, 1024b, 1218b, 1514b
These packet sizes are used to determine the maximum throughput for the firewall DUT per fixed packet size
Test Duration Every test duration runs for at least 10 minutes
Desired Result
For throughput, maximum offered load, expressed in either bits per second or packets per second, at which no packet loss is detected. For forwarding rate, expressed in either bits per second or packet per second, the device is observed to successfully forward to the correct destination interface in response to a specified offered load.
Key Measured Metrics
Statistic Relevance
Step Iteration Show how many steps are added in the load profile to achieve maximum throughput
Pass/Fail Status Show successful/unsuccessful transactions
Total packets offered Show total packets offered
Total packets forwarded Show total packets forwarded
Intended load Show attempted transactions
Offered load (if applicable) N/A
Forwarding rate Show forwarding rate
Analysis
The user should not see lost packets. The user should see the maximum throughput when running the test for each different packet size. When testing with different packet sizes, the user should see different throughput measurements and the firewall DUT configuration must remain the same.
7
CAS_002 Stateful traffic concurrent TCP connection capacity per
RFC 3511
Abstract
This test determines the maximum number of concurrent TCP connections supported through or with the firewall (DUT). This test also finds the maximum number of entries the DUT can store in its connection table. For this test, HTTP 1.1 must be used on both the client and server side. In the case of a proxy-based firewall, the clients must request at least two objects – one to measure the TCP establishment time (Time to SYN/ACK and Time to First Byte), the subsequent objects to evaluate the quality of the connection (Average URL Response Time). References: RFC 3511. For product verification and engineering.
Description
This test employs an iterative search algorithm to determine the maximum number of concurrent TCP connections supported through or with the DUT. For each iteration, the aggregate number of concurrent TCP connections attempted by the virtual clients is varied. The destination address is that of the server or that of the NAT proxy. The aggregate rate is defined by the connection attempt rate, and is attempted in a round-robin fashion.
To validate all connections, the virtual clients request an object using an HTTP 1.1 or higher GET request. The requests are initiated on each connection after all of the TCP connections have been established.
When testing proxy-based DUTs, the virtual clients request two objects using HTTP 1.1 or higher GET requests. The first GET request is required for connection time establishment measurements as specified. The second request is used for validation as previously mentioned. When comparing proxy and non-proxy based DUTs, the test is performed in the same manner. Between each iteration, it is recommended that the tester issue a TCP RST referencing each connection attempted for the previous iteration, regardless of whether or not the connection attempt was successful. The tester waits for aging time before continuing to the next iteration.
Relevance
For the networks that a firewall secures, often the value of the data and systems that they must help protect cost many times that of the firewall and the testing. Deploying a security
infrastructure without understanding its performance and security runs the risk of doing little to protect the network, especially if it also introduces apathy from a false sense of security. Among the reasons that show the criticality of testing firewalls:
Security at load: Often, security flaws do not appear until the network encounters a large load. Attacks can hide more easily within large amounts of traffic, potentially causing problems right when network downtime is most harmful.
Firewall behavior at load and at failure: Firewalls often exhibit different behaviors as they encounter increasing loads. Eventually, with enough traffic, the firewall will fail, providing
warning. Second, the failure state of the firewall is known—firewalls that fail closed will stop all traffic from passing (essentially a successful denial of service, or DoS, attack), and firewalls that fail open permit all traffic, which is a security failure.
Pre-deployment capacity planning: Deployment of a security infrastructure will most likely affect overall network performance. Testing the effect on network performance ensures that the increased security does not decrease performance beyond the levels acceptable for the business.
Version
1.0
Test Category
Testing Stateful Multiplay Benchmark Standards.
PASS
[ ] Performance [ ] Availability [ ] Security [x] Scale
Required Tester Capabilities
The test equipment must have the ability to generate simulated users. Each simulated user must have the ability to schedule full web pages with the ability to schedule specific HTTP objects within TCP tunnels in a predictable order, matched with a predictable source IP and MAC address source.
Topology
Test Procedure
1. Reserve two test ports.
2. Connect tester port 1 as the client side to the configured firewall capable DUT. Connect port 2 as the server side to the other side of DUT. Establish the link.
9
c. Select the performance test for the specific equipment that is used in this test. d. The goal is to open a connection, perform a GET, wait 30 seconds (the connection will
remain open unless the DUT has a TCP Timeout lower than this – but typically devices have at minimum 90 seconds TCP Timeout), perform another GET, wait 30 seconds, and so on until the ten GETs are complete.
e. If the maximum transactions per connection is set to 10, the connection is closed by a FIN/ACK. If the maximum transactions per connection is greater than the number of GETs in the action List, the connections is closed by an RST.
f. If the server-side maximum requests per connection is equal to the number of GETs in the client action list, and the client profile is set to a maximum request per connection higher than the number of GETs in the client action list, the connection termination is initiated server-side.
4. Begin Step 2 – Test the effect of load, object size, and duration. a. Client side:
i. HTTP version 1.1.
ii. HTTP GET Per SimUser is 10.
iii. Maximum connection per server is 1.
iv. Maximum transaction per connection is either 10 or 50. v. User think time is 30 seconds.
b. Server side:
i. HTTP version 1.1.
ii. Maximum requests per connection are either 64 or 10 (if client is set to more than 10).
iii. Server type is Irrelevant (only changes HTTP headers). iv. Object size is 64b, 512b, 1,024b, 10,240b.
5. End.
Variables & Relevance
Variable Relevance
Packet sizes – 64, 512, 1024, 10240
These packet sizes are used to determine the maximum throughput for the firewall DUT per fixed packet size
Test Duration Every test duration runs for at least 10 minutes
Desired Result
For maximum concurrent connections, the total number of TCP connections open for the last successful iteration performance in the search algorithm.
For minimum connection establishment time, the lowest TCP connection establishment time measured.
For maximum connection establishment time, the highest TCP connection establishment time measured.
For average connection establishment time, the mean of all measurements of connections establishment times.
Key Measured Metrics
Statistic Relevance
Object size Show size of object for each test iteration
Number of completed requests Show number of completed transactions (client side)
Number of completed responses Show number of completed transactions (server side)
Analysis
The user should not see lost packets. The user should see the maximum concurrent connections, minimum connection establishment time, maximum connection establishment time, average connection establishment time, aggregate connection establishment time, number of objects requested, and number of objects returned for each of the test iteration that uses different packet sizes. When testing with different packet sizes, the user should see different throughput measurements and the firewall DUT configuration must remain the same.
11
CAS_003 Determine the maximum TCP connection establishment
rate
Abstract
This test determines the maximum TCP connection establishment rate through or with the firewall (DUT). This test also finds the maximum rate at which the DUT can update its connection table. The test defines the aggregate number of TCP connections that must be established. HTTP 1.1 or higher must be used for this test for both clients and servers. The client and server must use the same HTTP version. The test also defines the number of bytes, excluding any bytes associated with the HTTP header, to be transferred in response to an HTTP 1.1 or higher GET request. References: RFC 3511, RFC 2647. For product verification and engineering.
Description
This test employs an iterative search algorithm to determine the maximum rate at which the DUT can accept TCP connection requests. For each iteration, the aggregate rate at which TCP
connection requests are attempted by the virtual clients is varied. The destination address is that of the server or that of the NAT proxy. The aggregate number of connections, defined by number of connections, is attempted in a round-robin fashion.
The same application-layer object transfers required for validation and establishment time measurements as described in the concurrent TCP connection capacity test is performed. Between each iteration, it is recommended that the tester issue a TCP RST referencing each connection attempted for the previous iteration, regardless of whether or not the connection attempt was successful. The tester waits for aging time before continuing to the next iteration.
Relevance
For the networks that a firewall secures, often the value of the data and systems that they must help protect cost many times that of the firewall and the testing. Deploying a security
infrastructure without understanding its performance and security runs the risk of doing little to protect the network, especially if it also introduces apathy from a false sense of security. Among the reasons that show the criticality of testing firewalls:
Security at load: Often, security flaws do not appear until the network encounters a large load. Attacks can hide more easily within large amounts of traffic, potentially causing problems right when network downtime is most harmful.
Firewall behavior at load and at failure: Firewalls often exhibit different behaviors as they encounter increasing loads. Eventually, with enough traffic, the firewall will fail, providing valuable insights. First, the conditions and behavior leading up to failure are now known, giving the security administrator things to look for in production and providing a useful pre-failure warning. Second, the failure state of the firewall is known—firewalls that fail closed will stop all traffic from passing (essentially a successful denial of service, or DoS, attack), and firewalls that fail open permit all traffic, which is a security failure.
Version
1.0
Test Category
Testing Stateful Multiplay Benchmark Standards.
PASS
[ ] Performance [ ] Availability [ ] Security [x] Scale
Required Tester Capabilities
Specific Spirent features that enable this test case and emphasize the Spirent differentiators and branding.
Topology
Test Procedure
1. Reserve two test ports.
2. Connect tester port 1 as the client side to the configured firewall capable DUT. Connect port 2 as the server side to the other side of the DUT. Establish the link.
3. Begin Step 1 – Configure the connections per second performance test.
13
e. To maximize the performance of the tester, this test sends 10 HTTP level 1 GET requests from each SimUser.
f. Each TCP connection accepts one HTTP transaction as the maximum transaction, therefore each of the SimUsers generates 10 TCP connections sequentially. 4. Begin Step 2 – Test the effect of load, object size, and duration.
a. Client side:
i. HTTP version 1.1.
ii. HTTP GET per SimUser is 10.
iii. A maximum connection per server is 1. iv. Maximum transaction per connection is 1.
v. User think time 30 seconds. b. Server side:
i. HTTP version 1.1.
ii. A maximum request per connection is 1.
iii. Server type is Irrelevant (only changes HTTP headers). iv. Object size is 64b, 512b, 1,024b, 10,240b.
5. End.
Variables & Relevance
Variable Relevance
Packet sizes – 64b, 512b, 1024b, 10240b
These packet sizes are used to determine the maximum throughput for the firewall DUT per fixed packet size
Test Duration Every test duration runs for at least 10 minutes
Desired Result
For highest connection rate, in connections per second, the rate for which all connections successfully opened in the search algorithm.
For minimum connection establishment time, the lowest TCP connection establishment time measured.
For maximum connection establishment time, the highest TCP connection establishment time measured.
For average connection establishment time, the mean of all measurements of connection establishment times.
For aggregate connection establishment time, the total of all measurements of connection establishment times.
Key Measured Metrics
Statistic Relevance
Number of object requested Show the total number requested objects
Analysis
The user should not see lost packets. The user should see the maximum TCP connections rate, minimum connection establishment time, maximum connection establishment time, average connection establishment time, aggregate connection establishment time, number of object requested, and number of object returned for each of the test iteration that uses different packet sizes. When testing with different packet sizes, the user should see different throughput measurements and the firewall DUT configuration must remain the same.
15
CAS_004 Determine the maximum TCP connection tear down rate
Abstract
This test determines the maximum TCP connection teardown rate through or with the firewall (DUT). This test defines the number of TCP connections that are attempted to be torn down. It also defines a method for closing TCP connections. The test must be performed with either a three-way or four-way handshake. In a four-way handshake, each side sends separate FIN and ACK messages. In a three-way handshake, one side sends a combined FIN/ACK message upon receipt of a FIN. The test also defines whether closing of connections is to be initiated from the client or from the server. References: RFC 3511, RFC 2647. For product verification and engineering.
Description
This test employs an iterative search algorithm to determine the maximum TCP connection tear down rate supported by the DUT. The test iterates through different TCP connection tear down rates with a fixed number of TCP connections.
In the case of proxy based DUTs, the DUT receives the ACK in response to issuing a FIN packet to close its side of the TCP connection. For validation purposes, the virtual client or server,
whichever is applicable, may verify that the DUT received the final ACK by re-transmitting the final ACK. A TCP RST should be received in response to the retransmitted ACK.
Between each iteration, it is RECOMMENDED that the virtual clients or servers, whichever is applicable, issue a TCP RST referencing each connection which was attempted to be torn down, regardless of whether or not the connection tear down attempt was successful. The test waits for aging time before continuing to the next iteration.
Relevance
For the networks that a firewall secures, often the value of the data and systems that they must help protect cost many times that of the firewall and the testing. Deploying a security
infrastructure without understanding its performance and security runs the risk of doing little to protect the network, especially if it also introduces apathy from a false sense of security. Among the reasons that show the criticality of testing firewalls:
Security at load: Often, security flaws do not appear until the network encounters a large load. Attacks can hide more easily within large amounts of traffic, potentially causing problems right when network downtime is most harmful.
Firewall behavior at load and at failure: Firewalls often exhibit different behaviors as they encounter increasing loads. Eventually, with enough traffic, the firewall will fail, providing valuable insights. First, the conditions and behavior leading up to failure are now known, giving the security administrator things to look for in production and providing a useful pre-failure warning. Second, the failure state of the firewall is known—firewalls that fail closed will stop all traffic from passing (essentially a successful denial of service, or DoS, attack), and firewalls
Pre-deployment capacity planning: Deployment of a security infrastructure will most likely affect overall network performance. Testing the effect on network performance ensures that the increased security does not decrease performance beyond the levels acceptable for the business.
Version
1.0
Test Category
Testing Stateful Multiplay Benchmark Standards.
PASS
[ ] Performance [ ] Availability [ ] Security [x] Scale
Required Tester Capabilities
The test equipment must have the ability to generate simulated users. Each simulated user must have the ability to schedule full web pages with the ability to schedule specific HTTP objects within TCP tunnels in a predictable order, matched with a predictable source IP and MAC address source.
Topology
17
a. Enter the management IP address of the equipment that is used for this test for the client port.
b. Enter the management IP address of the equipment that is used for this test for the server port.
c. Select the performance test for the specific equipment that is used in this test. d. Make sure the test is set up to use HTTP 1.1 for both client and server.
e. To maximize the performance of the tester, this test sends 10 HTTP level 1 GET requests from each SimUser.
f. Each TCP connection accepts one HTTP transaction as the maximum transaction, therefore each of the SimUser will generate 10 TCP connections sequentially. 4. Begin Step 2 – Test the effect of load, object size, and duration.
a. Client side:
i. HTTP version 1.1.
ii. HTTP GET per SimUser is 10.
iii. Maximum connections per server is 1. iv. Maximum transaction per connection is 1.
v. User think time 30 seconds. b. Server side:
i. HTTP version 1.1.
ii. Maximum requests per connection is 1.
iii. Server type is Irrelevant (only changes HTTP headers). iv. Object size is 64b, 512b, 1,024b, 10,240b.
5. End.
Variables & Relevance
Variable Relevance
Packet sizes – 64b, 512b, 1024b, 10240b
These packet sizes are used to determine the maximum throughput for the firewall DUT per fixed packet size
Test Duration Every test duration runs for at least 10 minutes
Desired Result
For highest connection tear down rate, the highest rate in connections per second, for which all TCP connections were successfully torn down in the search algorithm.
For minimum connection tear down time, the lowest TCP connection tear down time measured.
For maximum connection tear down, the highest TCP connection tear down time measured. For average connection tear down time, the mean of all measurements of connection tear down time.
For aggregate connection tear down time, the total of all measurements of connection tear down time.
Key Measured Metrics
Statistic Relevance
Number of connections Show the total number of connections
Aging time Show the aging time
Close method Show 3-way or 4-way handshake
Close direction Show close of connections are initiated from either client or server
Analysis
The user should not see lost packets. The user should see the maximum TCP connections tear down rate, minimum connection tear down time, maximum connection tear down time, average connection tear down time, aggregate connection tear down time, number of connections, aging time, close method, and close direction for each of the test iteration which uses different packet sizes. When testing with different packet sizes, the user should see different throughput measurements and the firewall DUT configuration must remain the same.
19
CAS_005 Measure the BBT external to external basic throughput
Abstract
This test is delivered by Broadband Testing (BBT) and is focused on securing the virtual data center in a cloud computing environment. This test case determines a baseline performance including the maximum realistic throughput of the DUT under sustained attack. For product verification and engineering.
Description
This test determines maximum throughput. The DUT is connected to the tester via a 10 GbE connection and also connected to the main test network via 1 Gbps connections to the virtual elements of the test bed.
Relevance
Measures throughput of the external-to-external throughput of a DUT Measures performance of virtualized applications
Version
1.0
Test Category
Testing Security and Cloud Services.
PASS
[x] Performance [ ] Availability [ ] Security [ ] Scale
Required Tester Capabilities
Ability to import the BBT test file
Support for server transactions profiles for different packet sizes
Topology
Test Procedure
1. Reserve two test ports.
2. Connect tester port 1 as the client side to the DUT. Connect port 2 as the server side to the other side of DUT. Establish the link.
3. Begin Step 1 – Import the test file (spf) from BBT
a. Select the Ext_to_Ext_Final_PlusAttacks_0001 test from SBFinal Project b. Set the client side port to the network that is currently connected. c. Set the server side port to the network that is currently connected. 4. Begin Step 2 – Test the effect of load, packet size, and duration.
a. Client side:
i. HTTP version 1.1. ii. HTTP GET per SimUser.
iii. Maximum connections per server 2. iv. Maximum requests per connection 50.
v. User thinks time 0 seconds.
vi. User-based load profile for http traffic (SimUsers/sec) and threats traffic (SimUsers/sec).
b. Server side:
i. HTTP version 1.1.
ii. Maximum requests per connection 64.
iii. Server type irrelevant (only changes HTTP headers). iv. Packet size 100KB.
5. Begin Step 3 – Run the test 6. End.
21
Variables & Relevance
Variable Relevance
Packet sizes – 100KB This packet size is intended to determine the maximum throughput for the DUT
Stateful Threats (Attack List) – 16 threats were used
There are 16 stateful threats that are used to determine whether the DUT is able to block them and still sustain the throughput.
Test Duration Each test duration runs for at least 12 minutes.
Desired Result
For throughput, the maximum offered load, at which no packet loss is detected.
Key Measured Metrics
Statistic Relevance
Step Iteration Show how many steps are added in the load profile to achieve maximum throughput
Pass/Fail Status Show successful/unsuccessful transactions
Intended load Show attempted transactions
Exploitation Results Show attack type, client success exploit ratio
Throughput Results (Incoming and Outgoing Traffic)
Show throughput level of DUT
Analysis
The user should not see lost packets. The user should see the maximum throughput when running the test using 100KB objects and while blocking threat traffic.
CAS_006 Measure the BBT external to internal (virtual) max
throughput
Abstract
This test is delivered by Broadband Testing (BBT) and is focused on securing the virtual data center in a cloud computing environment. This test case determines the performance from external (physical) to internal (virtual) environments including the maximum realistic throughput of the DUT. For product verification and engineering.
Description
This test determines the maximum throughput when generating traffic to move from the physical to the virtual elements of the test bed.
Relevance
Determines the maximum virtual max throughput.
Version
1.0
Test Category
Testing Security and Cloud Services.
PASS
[ ] Performance [x] Availability [ ] Security [ ] Scale
Required Tester Capabilities
Ability to import the BBT test file
Support for server transactions profiles for different packet sizes
23
Topology
Test Procedure
1. Reserve three test ports (1 client and 2 servers).
2. Connect the physical tester to the DUT. Run the virtual tester in a VM on the DUT. Establish the link.
3. Begin Step 1 – Import the test file (spf) from BBT.
a. Select the Ext_to_Int_Bandwidth_1200b_Post test from SBFinal Project. b. Set the client port to the network that is currently connected.
c. Set the server ports port to the network that is currently connected. 4. Begin Step 2 – Test the effect of load, packet size, and duration.
a. Client side:
i. HTTP version 1.1.
ii. HTTP GET and POST per SimUser. iii. Maximum connections per server 2. iv. Maximum requests per connection 50.
v. User think time 0 seconds.
vi. Global load profile for http traffic (SimUsers/sec). b. Server side:
i. HTTP version 1.1.
ii. Maximum requests per connection 64.
iii. Server type irrelevant (only changes HTTP headers). iv. Packet size 1200 bytes.
5. Begin Step 3 – Run the test. 6. End.
Variables & Relevance
Variable Relevance
Packet sizes for GET – 500, 1024, 1200 bytes
This packet size is used to determine the maximum throughput for the DUT.
Packet sizes for POST – 500, 1024, 1200 bytes
This packet size is used to determine the maximum throughput for the DUT.
Desired Result
For throughput, the maximum offered load, at which no packet loss is detected. Sample results below:
External To Internal Tests GET GET GET POST POST POST
Packet size (bytes) 500 1024 1200 500 1024 1200
Incoming (kbps) 280730 433723 451023 251535 304764 308076
Outgoing (kbps) 77205 69064 62883 294369 333936 333234
Transactions Per Second 49189 43779 39860 47901 32915 28370
Key Measured Metrics
Statistic Relevance
Step Iteration Show how many steps are added in the load profile to achieve maximum throughput
Pass/Fail Status Show successful/unsuccessful transactions
Intended load Show attempted transactions
Throughput Results
(Incoming and Outgoing Traffic)
Show throughput level of DUT
Analysis
In this test, examine the packet size for both GET and POST methods. Compare the measured results to the desired results in the above table. Report the results as a table comparing the differences.
25
CAS_007 Measure the BBT internal (virtual) to internal (virtual)
maximum throughput
Abstract
This test is delivered by Broadband Testing (BBT) and is focused on securing the virtual data center in a cloud computing environment. This test case determines the performance from internal (virtual) to internal (virtual) environments including the maximum realistic throughput of the DUT. For product verification and engineering.
Description
This test determines the maximum throughput when generating traffic to move from the internal (virtual) to the virtual elements of test bed. This test determines whether the virtual controller in the DUT affects performance, by generating HTTP traffic to achieve the maximum level of throughput in the virtual environments.
Relevance
Measures the virtual to virtual maximum performance
Version
1.0
Test Category
Testing Security and Cloud Services.
PASS
[x] Performance [ ] Availability [ ] Security [ ] Scale
Required Tester Capabilities
Ability to import the BBT test file
Support for server transactions profiles for different packet sizes
Topology
Test Procedure
1. Reserve three test ports (1 client and 2 servers).
2. Connect the physical tester to the DUT. Run the virtual tester in a VM on the DUT. Establish the link.
3. Begin Step 1 – Import the test file (spf) from BBT. a. Select the Int_to_Int test from SBFinal Project.
b. Set the client port to the network that is currently connected. c. Set the server ports to the network that is currently connected. 4. Begin Step 2 – Test the effect of load, packet size, and duration.
a. Client side:
i. HTTP version 1.1. ii. HTTP GET per SimUser.
iii. Maximum connections per server 2. iv. Maximum requests per connection 50.
v. User think time 0 seconds.
vi. Global load profile for http traffic (SimUsers/sec). b. Server side:
i. HTTP version 1.1.
ii. Maximum requests per connection 64.
iii. Server type irrelevant (only changes HTTP headers). iv. Packet size 100KB.
5. Begin Step 3 – Run the test.
a. Run with the virtual controller redirector in the DUT disabled. Traffic will pass straight-through.
27
Variables & Relevance
Variable Relevance
Packet sizes for GET – 100KB, 500 bytes, 1024 bytes, and 1200 bytes
This packet size is used to determine the maximum throughput for the DUT.
Test Duration Each test duration runs for at least 12 minutes.
Desired Result
For throughput, the maximum offered load, at which no packet loss is detected. Sample results below:
External To Internal Tests GET GET GET
Packet size (bytes) 500 1024 1200
Incoming (kbps)
64404
111952 126393
Outgoing (kbps)17831
17886
17715
Transactions Per Second11348
11342
11205
Key Measured Metrics
Statistic Relevance
Step Iteration Show how many steps are added in the load profile to achieve maximum throughput
Pass/Fail Status Show successful/unsuccessful transactions
Intended load Show attempted transactions
Throughput Results (Incoming and Outgoing Traffic)
Show throughput level of DUT
Transactions per second Show how many transactions per second are successfully for different packet size
Analysis
The user should not see lost packets. The user should see the maximum throughput for each test iteration. When testing with different packet sizes, the user should see different throughput measurements and the DUT configuration must remain the same. The user should see performance improve significantly with a higher performance server.
CAS_008 Measure performance of HTTP over VPLS over BGP in the
cloud
Abstract
HTTP goodput is a critical core metric when analyzing a service provider core network. This test allows the user to measure HTTP goodput ranges over a discrete number of VPLS tunnels. Without understanding the range of goodput across the cloud, the service provider will not be able to properly design and deploy the proper network scale of each device in the network. References: RFC2616 Hyper Text Markup Languages. For functional and system performance testing.
Description
HTTP goodput is the measure of delivered bandwidth to the HTTP stack with all network impairments taken into consideration. Since traffic in the service provider cloud is tunneled and backhauled through the cloud, impairments in the core can have a direct impact on the overall user experience of web based traffic, end to end. Impairments such as network induced jitter, excessive latency distribution, sequencing errors, and errors can all contribute to poor network traffic.
Relevance
Understanding goodput ranges in a specific network environment is a key attribute of end user experience.
Version
1.0
Test Category
Performance Test.
PASS
[x] Performance [ ] Availability [ ] Security [ ] Scale
Required Tester Capabilities
The test equipment should have the ability the ability to dynamically setup control plane routing and BGP and VPLS. Over VPLS, the user should have the ability to make dynamic changes to HTTP traffic. The tester should have the ability to coordinate and trend responses.
29
Topology
Test Procedure
1. Reserve tester ports A and B. connect the ports to the DUT and bring up the links. 2. If P router emulation is selected, setup desired number of P routers per port.
a. Establish one EBGP peering session per P router to the DUT.
3. Setup the desired number of PE router per tester port or emulated P router. a. Emulate 1 CE router per PE router.
b. Setup a host clouds behind each CE router. 4. Advertise though BGP all host cloud subnets.
5. Establish one LDP VPLS tunnel from each CE emulated port on tester A to each emulated CE router on tester port B. Verify all tunnels establish.
6. On tester port B, set up HTTP Servers with the desired HTTP return object size. Start all HTTP servers.
7. On the tester port A, setup one HTTP client per host. Each client should be connected to a unique emulated HTTP server.
8. For each client, set the following load specifications:
a. Ramp from 0 kbps to (desired peak aggregate bandwidth / tester port A HTTP host count) in 30 seconds.
b. Set steady state for (total test time – 100 seconds). c. Ramp down to 0 kbps in 30 seconds.
9. Start all HTTP clients.
Variables & Relevance
Variable Relevance
Emulate P Routers? Ask user is they wish to Emulate P Routers or just PE routers. Default=Yes
Number of P Routers Per Port How many P Routers to Emulate per Port. Default=5
PE Routers or PE Routers per Port?
Defines how many PE Routers to emulate per tester port or per P Router
Tester Port A Starting BGP AS Starting AS Number of Tester Port A
Tester Port A Starting IP Address Starting IP Address of the First P or PE Router
Tester Port A Netmask Classless Netmask of Tester Port A
Tester Port A Starting AS Starting AS of the P or PE Router
Tester Port A Starting BGP AS Starting AS Number of Tester Port A
Tester Port A DUT IP Address Tester Port B DUT IP Address
Tester Port B Starting IP Address Starting IP Address of the First P or PE Router
Tester Port B Netmask Classless Netmask of Tester Port A
Tester Port B Starting AS Starting AS of the P or PE Router
Tester Port B DUT IP Address Tester Port B DUT IP Address
Hosts per CE Cloud Number of Hosts per CE Cloud. Default=5
HTTP Object Size Object Size to Return. Default=384 bytes
Test Duration Duration of the test. Default=600 Seconds
Peak Aggregate HTTP Bandwidth Peak Aggregate HTTP Bandwidth. Default=150 Mbps
Desired Result
The DUT should be able to deliver the offered bandwidth. The variance in goodput per individual host should be zero.
Key Measured Metrics
Statistic Relevance
Aggregate HTTP Goodput Total HTTP achieved globally
Per Host HTTP Goodput Per Host HTTP goodput
Analysis
To facilitate analysis, the following tables and charts should be created.
The offered and measured bandwidth and goodput should be displayed as a bar chart
comparison, there should also be a table presenting direct numbers and difference in kbps and percentage.
The next chart should be offered bandwidth vs. aggregate goodput every 10 seconds.
There should be a chart showing per host goodput with the X-Axis representing VPLS tunnel and associated transactions behind each tunnel, and the Y-Axis repressing measured goodput as a
31
CAS_009 BBT – External to internal (virtual): scalability and
application availability
Abstract
This test is delivered by Broadband Testing (BBT) and focuses on securing the virtual data center. This test case determines application availability under load from external to internal (virtual) environments and scalability of server performance.
Description
This test determines application availability when generating traffic to move from external to virtual elements of the BBT test bed. This test also determines scalability of server performance and response times while under high load.
Relevance
Performance (outright throughput and application availability/latency) Scalability
Security Efficiency
Version
1.0
Test Category
Testing Security and Cloud Services
PASS
[x] Performance [x] Availability [x] Security [x] Scale
Required Tester Capabilities
Import BBT test file
Server transactions profile support for different packet sizes
Topology
Test Procedure
1. Reserve three test ports (1 client and 2 servers)
2. Connect cabling to the DUT. Connect an Avalanche 3100 and an Avalanche Virtual (running on a VM) as the physical and virtual components. Also connect to a management server (which consists of an IPS appliance, software running on VMs under VMware, vController, SMS management server, and management software). Establish link.
3. Begin Step 1 – Import the test file (spf) from BBT.
a. Select the Ext_to_Int_CC and Ext_to_Int_CPS tests from SBFinal Project.
b. From the client/port tab, change to the port to the network that is currently connected. c. From the server/port tab, change to the port to the network that is currently connected. 4. Begin Step 2 – Test the effect of load, packet size, and duration.
a. Client side:
i. HTTP version 1.0 and HTTP 1.1 persistent. ii. HTTP GET Per SimUser.
iii. Maximum Connections Per Server 2. iv. Maximum Requests per Connection 50.
v. User Think Time 0 seconds.
vi. Global load profile for http traffic (SimUsers and cps). b. Server side:
i. HTTP version 1.1.
ii. Maximum Requests per Connection 64.
iii. Server Type Irrelevant (only changes HTTP headers). iv. Packet size 500, 1024, and 1200 bytes.
33
Variables & Relevance
Variable Relevance
Packet sizes for GET – 500, 1024, 1200 bytes
This packet size determines the maximum throughput for the DUT.
Packet sizes for POST – 500, 1024, 1200 bytes
This packet size determines the maximum throughput for the DUT.
Test Duration Every test duration runs for at least 12 minutes
Desired Result
The result of this external-to-internal environment test demonstrates that the firewall is able to scale with more powerful servers in sub milliseconds using different packet sizes for virtual machines and while sustaining maximum throughput and application availability under load.
External To Internal Tests GET GET GET POST POST POST
Packet size (bytes) 500 1024 1200 500 1024 1200
Server Response Time (ms) 0.324 0.297 0.265 0.311 0.374 0.475
Key Measured Metrics
Statistic Relevance
Step Iteration Show how many steps are added in the load profile to achieve maximum throughput.
Pass/Fail Status Show successful/unsuccessful transactions.
Intended load Show attempted transactions.
Server Response Time (ms) Show server response time.
Throughput Results (Incoming and Outgoing Traffic)
Show throughput level of DUT.
Analysis
The user should not see lost packets. The user should see the server response time reported in sub milliseconds and the maximum throughput for each of the test iterations that uses different packet sizes. When testing with different packet sizes the user should see different server response times in sub milliseconds and throughput measurements and the DUT configuration must remain constant. The user should see the performance improve significantly with more powerful servers for the virtual end for both server response time and throughput
CAS_010 BBT – Internal (virtual) to internal (virtual): scalability
and application availability in a virtual environment
Abstract
This test is delivered by Broadband Testing (BBT) and focuses on securing the virtual data center. This test case determines application availability under load from internal (virtual) to internal (virtual) environments and scalability of server performance.
Description
This test determines application availability when generating traffic to move from internal to virtual elements of the BBT test bed. This test also determines the scalability of server performance and response time while under high load.
Relevance
Performance (outright throughput and application availability/latency) Scalability
Security Efficiency
Version
1.0
Test Category
Testing Security and Cloud Services
PASS
[x] Performance [x] Availability [x] Security [x] Scale
Required Tester Capabilities
Import BBT test file
Server transactions profile support for different packet sizes
35
Topology
Test Procedure
1. Reserve three test ports (1 client and 2 servers)
2. Connect cabling to the DUT. Connect Avalanche virtual (running on a VM) as the virtual components to management server (which consists of IPS appliance, software running on VMs under VMware, vController, SMS management server, and management software). Establish link.
3. Begin Step 1 – Import the test file (spf) from BBT.
a. Select the Int_to_Int_Bandwidth_1200b_0001, Int_to_Int_Bandwidth_1k_0001, and Int_to_Int_Bandwidth_500_0001 tests from SBFinal Project
b. From the client/port tab, change to the port to the network that is currently connected. c. From the server/port tab, change to the port to the network that is currently connected. 4. Begin Step 2 – Test the effect of load, packet size, and duration.
a. Client side:
i. HTTP version 1.0 and HTTP 1.1 persistent. ii. HTTP GET Per SimUser.
iii. Maximum Connections Per Server 2. iv. Maximum Requests per Connection 50.
v. User Think Time 0 seconds.
vi. Global load profile for http traffic (SimUsers and cps). b. Server side:
i. HTTP version 1.1.
ii. Maximum Requests per Connection 64.
iii. Server Type Irrelevant (only changes HTTP headers). iv. Packet size 500, 1024, and 1200 bytes.
5. Begin Step 3 – Running the test.
Variables & Relevance
Variable Relevance
Packet sizes for GET – 500, 1024, 1200 bytes
This packet size determines the maximum throughput for the DUT.
Test Duration Every test duration runs for at least 12 minutes
Desired Result
The result of this external-to-internal environment test demonstrates that the firewall is able to scale with more powerful servers in sub milliseconds using different packet sizes for virtual machines and while sustaining maximum throughput and application availability under load.
External To Internal Tests GET GET GET
Packet size (bytes) 500 1024 1200
Server Response Time (ms) 0.58 0.564 0.811
Key Measured Metrics
Statistic Relevance
Step Iteration Show how many steps are added in the load profile to achieve maximum throughput.
Pass/Fail Status Show successful/unsuccessful transactions.
Intended load Show attempted transactions.
Server Response Time (ms) Show server response time.
Throughput Results (Incoming and Outgoing Traffic)
Show throughput level of DUT.
Analysis
The user should not see lost packets. The user should see the server response time reported in sub milliseconds and the maximum throughput for each of the test iterations that uses different packet sizes. When testing with different packet sizes the user should see different server response times in sub milliseconds and throughput measurements and the DUT configuration must remain constant. The user should see the performance improve significantly with more powerful servers for the virtual end for both server response time and throughput
37
CAS_011 BBT – Internal (virtual) to internal (virtual): security –
attacking the virtual environment
Abstract
This test is delivered by Broadband Testing (BBT) and is focused on securing the virtual data center. This test case determines performance, including the maximum realistic throughput in the virtual environment, from internal (virtual) to internal (virtual) while under sustained attack.
Description
This test determines the maximum throughput within the virtual environment while under a series of different attacks. This test uses only the internal (virtual) environment and is setup using Avalanche virtual targeting vController directly with a combination of http traffic load generation and injected threat traffic, first based on CodeRed II followed by several different stateful attacks.
Relevance
Performance (outright throughput and application availability/latency) Scalability
Security Efficiency
Version
1.0
Test Category
Testing Security and Cloud Services
PASS
[x] Performance [x] Availability [x] Security [x] Scale
Required Tester Capabilities
Import BBT test file
Server transactions profile support for different packet sizes
Topology
Test Procedure
1. Reserve two test ports.
2. Connect cabling to the DUT. Connect vController and management server and Avalanche virtual (running on a VM) as the virtual and virtual component (which consists of IPS appliance, software running on VMs under VMware, vController, SMS management server, and management software). Establish link.
3. Begin Step 1 – Import the test file (spf) from BBT.
a. Select the Int_to_Int_Final_PlusAttacks_10G test from SBFinal Project.
b. From the client/port tab, change to the port to the network that is currently connected. c. From the server/port tab, change to the port to the network that is currently connected. 4. Begin Step 2 – Test the effect of load, packet size, and duration.
a. Client side:
i. HTTP version 1.0 and HTTP 1.1 persistent. ii. HTTP GET Per SimUser.
iii. Maximum Connections Per Server 2. iv. Maximum Requests per Connection 50.
v. User Think Time 0 seconds.
vi. User-based load profile for http traffic (SimUsers/sec) and threats traffic (SimUsers/sec)
b. Server side:
i. HTTP version 1.1.
ii. Maximum Requests per Connection 64.
39
c. Repeat the same test, generate HTTP traffic over 10 minutes, then inject multiple stateful attacks for 80 seconds into the test.
d. Repeat the same test, generate HTTP traffic over 10 minutes, then inject multiple stateful attacks for 80 seconds into the test. On the IPS DUT, add a rule to the firewall to block a batch of clients based on IP address.
6. End.
Variables & Relevance
Variable Relevance
Packet sizes – 100KB This packet size determines the maximum throughput for the DUT
Stateful Threats (Attack List) Stateful threats
Test Duration Every test duration runs for at least 90 seconds or 10 minutes
Desired Result
The first test generates the maximum throughput of HTTP traffic over 90 seconds runs and attacks are injected, the DUT in the virtual environment successfully blocks the bad traffic with no reduction in overall maximum throughput.
The second test generates the maximum throughput of HTTP traffic for over 90 seconds and multiple stateful attacks are injected which were contained within the signature database, the DUT in the virtual environment successfully block all the bad traffic. Since the attacks attack across a saturated link – VM to VM – the result of this test should show the vController’s ability to successfully block with no reduction in overall maximum throughput.
The third test should generate the maximum throughput of HTTP traffic and the DUT should block all bad traffic.
Key Measured Metrics
Statistic Relevance
Step Iteration Show how many steps are added in the load profile to achieve maximum throughput.
Pass/Fail Status Show successful/unsuccessful transactions.
Intended load Show attempted transactions.
Server Response Time (ms) Show server response time.
Throughput Results (Incoming and Outgoing Traffic)
Show throughput level of DUT.
Analysis
The user should not see lost packets. The user should see the server response time reported in sub milliseconds and the maximum throughput for each of the test iterations that uses different packet sizes. When testing with different packet sizes the user should see different server response times in sub milliseconds and throughput measurements and the DUT configuration must remain constant. The user should see the performance improve significantly with more
CAS_012 BBT – External to internal (virtual): security – combined
traffic and attacks
Abstract
This test is delivered by Broadband Testing (BBT) and is focused on securing the virtual data center. This test case combines 10Gbps traffic from Avalanche 3100 (external) to the firewall and 1 Gbps traffic within the virtual environment on the vController to determine a baseline
performance including the maximum realistic throughput of the DUT under sustained attack.
Description
This test determines the maximum throughput from an external to a virtual environment. The DUT is connected to a Spirent 3100 via a 10GbE connection and is also connected to the main test network via 1 Gbps connections to the virtual elements of the test bed with multiple stateful threats on the vController. The test bed is designed to run at 1 GbE and 10GbE (in the physical space) test options to emulate a physical or virtual data center environment.
Relevance
Performance (outright throughput and application availability/latency) Scalability
Security Efficiency
Version
1.0
Test Category
Testing Security and Cloud Services
PASS
[x] Performance [x] Availability [x] Security [x] Scale
Required Tester Capabilities
Import BBT test file
Server transactions profile support for different packet sizes
41
Topology
Test Procedure
1. Reserve two test ports.
2. Connect cabling to the DUT. Connect Avalanche 3100 10GigE directly to the Firewall DUT and Avalanche virtual 1GigE (running on a VM) as the physical and virtual components, also connect to management server (which consists of IPS appliance, software running on VMs under VMware, vController, SMS management server, and management software). Establish link.
3. Begin Step 1 – Import the test file (spf) from BBT. a. Select the Threats_test test from SBFinal Project.
b. From the client/port tab, change to the port to the network that is currently connected. c. From the server/port tab, change to the port to the network that is currently connected. 4. Begin Step 2 – Test the effect of load, packet size, and duration.
a. Client side:
i. HTTP version 1.1 ii. HTTP GET Per SimUser.
iii. Maximum Connections Per Server 2. iv. Maximum Requests per Connection 50.
v. User Think Time 0 seconds.
vi. User-based load profile for http traffic (SimUsers/sec) and threats traffic (SimUsers/sec)
b. Server side:
i. HTTP version 1.1.
ii. Maximum Requests per Connection 64.
iii. Server Type Irrelevant (only changes HTTP headers). iv. Packet size 1024 bytes.
5. Begin Step 3 – Running the test.
Variables & Relevance
Variable Relevance
Packet sizes – 1024 bytes This packet size determines the maximum throughput for the DUT
Stateful Threats (Attack List) – 16 threats
There are 16 stateful threats to determine whether the DUT is able to block them and still sustain the throughput
Test Duration Every test duration runs for at least 10 minutes
Desired Result
For throughput, maximum offered load, at which no packet loss is detected.
The external and virtual environment generates around 4.5 Gbps of maximum throughput between the Avalanche 3100 10 GigE and the firewall DUT and multiple stateful threats are injected at the vController. The vController should be able to sustain maximum throughput while blocking threat traffic in addition to http traffic sending 1024 bytes of GET requests.
Key Measured Metrics
Statistic Relevance
Step Iteration Show how many steps are added in the load profile to achieve maximum throughput.
Pass/Fail Status Show successful/unsuccessful transactions.
Intended load Show attempted transactions.
Server Response Time (ms) Show server response time.
Throughput Results (Incoming and Outgoing Traffic)
Show throughput level of DUT.
Analysis
The user should not see lost packets. The user should see the server response time reported in sub milliseconds and the maximum throughput for each of the test iterations that uses different packet sizes. When testing with different packet sizes the user should see different server response times in sub milliseconds and throughput measurements and the DUT configuration must remain constant. The user should see the performance improve significantly with more powerful servers for the virtual end for both server response time and throughput
