Can Security Testing Be Simple?

(1)

Can Security Testing

Be Simple?

A Report of Spirent Avalanche NEXT

Featuring Palo Alto Networks PA-5020

Next Generation Firewall

A Broadband-Testing Report

(2)

First published February 2014 (V1.0) Published by Broadband-Testing Andorra Tel: +376 633010 E-mail: [email protected] Internet: HTTP://www.broadband-testing.co.uk 2014 Broadband-Testing

Please note that access to or use of this Report is conditioned on the following:

1. The information in this Report is subject to change by Broadband-Testing without notice.

2. The information in this Report, at publication date, is believed by Broadband-Testing to be accurate and reliable, but is not guaranteed. All use of and reliance on this Report are at your sole risk. Broadband-Testing is not liable or responsible for any damages, losses or expenses arising from any error or omission in this Report.

3. NO WARRANTIES, EXPRESS OR IMPLIED ARE GIVEN BY Broadband-Testing. ALL IMPLIED WARRANTIES,

INCLUDING IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT ARE DISCLAIMED AND EXCLUDED BY Testing. IN NO EVENT SHALL Broadband-Testing BE LIABLE FOR ANY CONSEQUENTIAL, INCIDENTAL OR INDIRECT DAMAGES, OR FOR ANY LOSS OF PROFIT, REVENUE, DATA, COMPUTER PROGRAMS, OR OTHER ASSETS, EVEN IF ADVISED OF THE

POSSIBILITY THEREOF.

4. This Report does not constitute an endorsement, recommendation or guarantee of any of the products (hardware or software) tested or the hardware and software used in testing the products. The testing does not guarantee that there are no errors or defects in the products, or that the products will meet your expectations, requirements, needs or

specifications, or that they will operate without interruption.

5. This Report does not imply any endorsement, sponsorship, affiliation or verification by or with any companies mentioned in this report.

6. All trademarks, service marks, and trade names used in this Report are the trademarks, service marks, and trade names of their respective owners, and no endorsement of, sponsorship of, affiliation with, or involvement in, any of the testing, this Report or Broadband-Testing is implied, nor should it be inferred.

(3)

BROADBAND-TESTING

Broadband-Testing is Europe’s foremost independent network testing facility and consultancy organisation for broadband and network infrastructure products.

Based in Andorra, Broadband-Testing provides extensive test demo facilities. From this base,

Broadband-Testing provides a range of specialist IT, networking and development services to vendors and end-user organisations throughout Europe, SEAP and the United States.

Broadband-Testing is an associate of the following:

Limbo Creatives (bespoke software development)

Broadband-Testing Laboratories are available to vendors and end-users for fully independent

testing of networking, communications and security hardware and software.

Broadband-Testing Laboratories operates an Approval scheme which enables products to be

short-listed for purchase by end-users, based on their successful approval.

Output from the labs, including detailed research reports, articles and white papers on the latest network-related technologies, are made available free of charge on our web site at

HTTP://www.broadband-testing.co.uk

Broadband-Testing Consultancy Services offers a range of network consultancy services including

(5)

EXECUTIVE SUMMARY

Our challenge was to gauge whether high-level performance testing has to be complex or, despite, the actual complexity of the testing beneath, can the "human" mechanics of the testing actually be simplified?

One angle is that simplicity eliminates the possibility of human error and cuts timescales and therefore costs. Equally, from a product launch perspective it accelerates launch speed, thereby maximising profit potential.

In order to put the theory to the test, we worked with Palo Alto Networks, who provided an Enterprise-level Next Generation Firewall (NGFW) in the form of the PA-5020 and Spirent Avalanche NEXT security performance tester.

We set up a test bed and ran a series of tests aimed at confirming Palo Alto Networks "marketing figures" for the PA-5020—the ability to run at 2Gbps with every firewall feature enabled, regardless of traffic type.

We set up a series of mixed application tests in order to mirror real-world usage. Real applications and protocols were used, not simulations, and this enabled us to emulate thousands of users on a real network.

From a performance perspective, we found that the PA-5020 lived up to its billing. The marketing figures were achieved in each test and we found that adding features one by one— IPS, Anti-Virus, Anti-Spyware—had zero impact on performance. System utilisation did increase, but there was always some spare headroom, even when theoretically maxed out. From the Spirent Avalanche NEXT perspective, we found the tests quick and easy to configure and rerun while making as many changes as we wished to elements such as the

application/protocol mix.

We were also able to make extensive use of the result/statistics provided by the auto-generated reports, examples of which you can see in this report.

Overall, we not only proved that the Palo Alto Networks PA-5020 performs as advertised, but we also proved that this level of testing need not be complex to setup and administer.

(6)

INTRODUCTION: SERIOUS STRESS TESTING COMBINED

WITH EASE OF USE—A CONTRADICTION IN TERMS?

For years, conceptually at least, there has been the general idea that high-level networking product testing is complicated.

That may be a great way to keep technical staff in a job, but it is in no way correct, in any sense! The reality is, and to some extent always has been, somewhat different. Yes, the test definitions, planning and execution need to be well thought-out and precise, but it doesn't mean that the test element itself needs to be complex. Of course, much of this boils down to the test tools being used. Ease of use is not a natural association with test tools, but time equals money very much is. So, the easier it is to use a test tool, quite simply the more cost efficient it is.

As IT budgets grow ever tighter, the need to deliver tried and tested solutions quickly, cheaply and reliably grows ever more important. But the big point about testing is that is never pays to cut corners; doing so only results in expensive secondary tasks or, at worst, disastrous sales and support issues. The question is then, is it possible to have a test environment that is both comprehensive at the highest test levels and easy to use? To answer this question, we used Spirent's "NEXT" version of its Avalanche test solution on the Palo Alto Networks PA-5020 firewall.

Basically, Avalanche NEXT is Spirent's existing Studio product, enhanced and ported to the Avalanche platform, which has long been at the heart of Spirent's most powerful, application-oriented test tools. The aim here is to provide a powerful engine that can be driven very easily by the NEXT interface. The results is a browser-based interface but the key is that every test option follows a near-identical setup procedure with as much drag-and-drop action as possible, rather than wading through tab after tab of setup options on multiple browser pages. So, in practice there are a near-unlimited range of test combinations available, especially as there is an online database of application/scenario downloads that is constantly being updated—the TestCloud database. In line with the ease of setting up the tests, the reports are self-generated and very clear—relatively concise but to the point—a fundamental of achieving quick but accurate test turn-arounds.

The question is: does this approach really work? In order to prove the concept, we've created a set of performance tests (application/protocol based) and partnered with Palo Alto Networks to carry out next generation firewall (NGNF) testing using Avalanche NEXT. So can simplicity and performance really be combined?

(7)

Spirent Avalanche NEXT: Feature Overview

So what exactly can you do with Avalanche NEXT?

Avalanche NEXT is designed to test the performance, scalability and security of contemporary

application-aware network infrastructures. It generates real high-performance user applications based on actual application scenarios for realistic security, load and functional testing. It is important to understand that testing application-aware infrastructures is very different to testing traditional network equipment, not least because application-aware devices look at the packet payload itself. For this reason it is important not to use "fake" traffic with the same packet payload over and over again, but authentic payloads based on actual application transactions and usage. In this way it is possible to verify the accuracy of application detection engines and policies by testing with thousands of popular application profiles, while at the same time evaluating the impact of security policies and QoS policies on application performance with mixed traffic attacks. In this way you can verify the ability of a security device to detect and mitigate thousands of known attacks. Equally, you can test the resilience of network devices and deployed protocols by verifying their ability to deal with millions of unexpected and malicious inputs and their ability to inspect traffic for malware, infected hosts, unwanted URLs and spam and take appropriate action.

(8)

Other applications of Avalanche NEXT include benchmark scalability and network capacity by

simulating up to hundreds of thousands of real users on the network and running "what-if" scenarios and measuring the performance impact of architecture or configuration changes. You can also validate proof-of-concept designs by testing with realistic mixes of application traffic.

Spirent defines the breadth of Avalanche NEXT in terms of four basic areas of coverage:

 Security

 Realism

 Performance

 Agility

Security: The ability to find and fix vulnerabilities quickly. Avalanche NEXT includes thousands of

attack profiles and vectors, so you can test any mix of attacks and applications to ensure network security including malware infected host emulation and malware binary transfer based security testing.

Realism: The ability to test your network, the traffic and the real situation. Avalanche NEXT

generates application traffic with authentic payloads based on actual usage for realistic security, load and functional testing. When testing application-aware devices, it is critically important that the application mix reflects real-world conditions from Layer 2-7. Avalanche NEXT enables you to create tests by capturing the interactions, plus you can quickly create custom tests for unique protocols and applications without scripting and effect smart remediation to shorten the time to fix vulnerabilities.

Performance: Extremely scalable performance. Avalanche NEXT is claimed to be the world’s highest

performing connection rate tester for HTTP, giving you the power to push devices and networks to their limits. It can deliver more than seven million HTTP connections/second using real applications.

Agility: The ability to test right now, not months from now. Avalanche NEXT includes the Spirent

TestCloud, giving you access to thousands of the latest performance and security tests and the ability to create new tests as soon as new applications or cutting edge protocols emerge. This includes thousands of user scenarios—from mobile handset-based apparitions to the latest in P2P file transfer. Moreover, in minutes you can capture your own network traffic and generate hundreds of automated tests from a single traffic capture.

In terms of what this means to different kinds of users, product development teams for example, it means they can complete testing more quickly and beat competitors to market and deliver a more robust and secure product. It also enables them to minimise field issues to reduce support costs and improve customer satisfaction.

For network engineers, it means they can deploy new applications and services with confidence, validate existing network services and infrastructure, find and fix issues before going to production and prevent fire-fighting while spending more time on projects that add value.

It means that Service Providers can ensure solutions will deliver the performance, scale and security required, complete the test phase of projects more quickly and identify performance and security issues more quickly for fast resolution. This will also minimise post-production issues and improve customer satisfaction.

(9)

Using the Avalanche NEXT Interface

The Avalanche NEXT main menu provides two basic sets of test options Audit/Performance Tests on the left-hand side of the screen and Protocol Tests on the right.

Figure 2 – The Spirent Avalanche NEXT Main Menu

The Audit/Performance tests include:

Cyber Security Assessment

The Cyber Security Assessment allows you to run over tens of thousands of modern and advanced attacks and malware (both binary transfer and infected host emulation).

Application Identification

Application Identification allows you to create high volumes of the latest mobile and cloud applications and security traffic patterns with many thousands of applications from the TestCloud database which is updated and available for download monthly ensuring you have the most popular and relevant applications and attacks for your testing.

Reliability Testing

Reliability Testingenables you to perform long duration soak tests with the TestCloud application load

to ensure solutions work at high capacity for long periods of time.

When setting up Audit/Performance tests you can make use of the TestCloud database of applications to create your own custom test by simply dragging and dropping which applications you want to include from the left of the screen; these might be Facebook, E-mail, Instant Messaging etc. and then defining the application mix for the test on the top right, as well as the duration of the test.

(10)

Figure 3 – Setting Up A Reliability Test Example

All actions are drag-and-drop or point-and-click based; a far cry from pages of tabular based entries required to set up a test.

The Protocol Tests which is the focus of our report include:

Throughput with Mixed Apps

With Throughput Mixed Apps you can create and run tests with preconfigured Enterprise traffic mix to achieve high throughput, application-based testing.

HTTP 1.1 Throughput

The HTTP 1.1 Throughput option allows you to create scalable HTTP tests with different object sizes to achieve line-rate throughput.

HTTP 1.0 Connection per Second

With a HTTP 1.0 Connection per Second you can create tests to achieve the industry's highest state generator with Extreme Scale and Performance module (ESP).

Fuzzing

Fuzzing lets you perform different service and protocol mutation scenarios through the Fuzz player to find vulnerabilities and test the reliability of implemented protocols with millions of test iterations created on-the-fly.

(11)

Again, when setting up Protocol tests, there is a significant amount of drag-and-drop action involved, such as for adding subnets (these are created in advance as profiles and you can create as many different scenarios as you want, and then mix and match them when setting up the tests) or for changing application mixes.

Figure 4 – Setting Up A Protocol (HTTP 1.1) Test Example

Single or multiple subnets can be used and test traffic patterns can be defined as pairs or backbone. Load specifications can be edited in terms of defined test metric (e.g. throughput or connections per second) pass/fail thresholds, test throughput height, ramp up/load profile, duration and other parameters.

Basically other than defining the basic subnets for the test, there is little in the way of additional "manual" configuration, other than possibly editing the default HTTP parameter settings, for example.

(12)

DEVICE UNDER TEST: PALO ALTO NETWORKS PA-5020

NGFW

For our testing, Palo Alto Networks kindly provided a PA-5020 NGFW.

Figure 5 – PA-5020 Firewall

This is an Enterprise firewall consisting of 12 Gigabit and eight 10Gigabit (SFP/SFP+) ports plus in and out of band management and single or dual SSD drives (up to 240GB capacity).

Key to the PA-5000 range is the ability to classify all applications on all ports at all times—what Palo Alto Networks calls "App-ID." This enables the firewall to:

 Identify the application, regardless of port, encryption (SSL or SSH) or evasive technique

employed.

 Use the application, not the port, as the basis for all safe enablement policy decisions: allow,

deny, schedule, inspect, and apply traffic shaping.

 Categorise unidentified applications for policy control, threat forensics, custom App-ID

creation, or packet capture for App-ID development.

Other features such as User-ID and GlobalProtect extend safe application enablement policies to any user at any location. The configuration includes agentless integration with Active Directory, LDAP, eDirectory Citrix and Microsoft Terminal Services and provides easy integration of firewall policies with NAC, 802.1X wireless, proxies and NAC solutions. The firewall is designed to block a range of known threats including exploits, malware and spyware, across all ports, regardless of common threat evasion tactics employed. It can also be configured to limit unauthorised transfer of files and sensitive data, and control non work-related web surfing, identify unknown malware, analyse it based on more than 100 malicious behaviours and then automatically create and deliver protection in the next content update.

The controlling element of the PA-5000 Series is PAN-OS™, a security-specific operating system that

natively classifies all traffic, inclusive of applications, threats and content, which then ties that traffic to the user, regardless of location or device type. The application, content, and user—in other words, the business elements that run your business—are then used as the basis of your security policies, resulting in an improved security posture and a reduction in incident response time.

In terms of capabilities, the PA-5020 is claimed to offer up to 5Gbps of firewall throughput, 2Gbps of threat prevention and VPN throughput, support a million sessions at a rate of 120,000 new sessions per second and supports up to 10,000 active policies.

(13)

What Exactly is a Next Generation Firewall (NGFW)?

A Next Generation Firewall is a device that identifies and securely enables applications, users and content moving across the network. Gartner created its own definition of an NGFW which states that, as a bare minimum, it should support in-line "bump-in-the-wire" configuration without disrupting network operations and act as a platform for network traffic inspection and network security policy enforcement, with the following minimum features:

Standard first-generation firewall capabilities: Use packet filtering, network address translation

(NAT), stateful protocol inspection, VPN capabilities, etc.

Integrated rather than merely co-located network intrusion prevention: Support

vulnerability-facing signatures and threat-vulnerability-facing signatures. The IPS interaction with the firewall should be greater than the sum of the parts, such as providing a suggested firewall rule to block an address that is continually loading the IPS with bad traffic. This exemplifies that, in the NGFW, it is the firewall correlates rather than the operator having to derive and implement solutions across consoles. Having high quality in the integrated IPS engine and signatures is a primary characteristic. Integration can include features such as providing suggested blocking at the firewall based on IPS inspection of sites only providing malware.

Application awareness and full stack visibility: Identify applications and enforce network security

policy at the application layer independent of port and protocol versus only ports, protocols and services. Examples include the ability to allow Skype use but disable file sharing within Skype or to always block GoToMyPC.

Extra firewall intelligence: Bring information from sources outside the firewall to make improved

blocking decisions, or have an optimised blocking rule base. Examples include using directory integration to tie blocking to user identity, or having blacklists and white lists of addresses. Also, provide support for upgrade paths for integration of new information feeds and new techniques to address future threats.

Outside of the Gartner NGFW definition, it is important to evaluate how traffic is classified and then inspected. The approach taken by Palo Alto Networks focuses on determining the application identity as the traffic hits the device. Regardless of which port the traffic is using, the application identity is determined and then used as the basis for all security policy decisions—including additional functions such as threat prevention and QoS.

Traditional firewall vendors are stateful inspection based, which means that an initial traffic decision is made based solely on the port, then any application control functions are performed after this, either using a separate blade, or the IPS engine, controlled by a separate policy or profile. For example, to control Facebook, you allow all port 80 traffic, then find the Facebook entry in the applications list and create a "block Facebook" rule. Palo Alto Networks claims that this "control-after-the-fact" approach weakens the deny-all-else premise upon which firewalls are built, in which case what happens to the other allowed applications that you are unaware of? Palo Alto Networks further claims that there can be significant performance degradation, policy reconciliation, and log visibility ramifications to this approach.

(14)

Regarding architecture, Palo Alto Networks has designed what it calls "a single pass parallel processing architecture" designed to addresses these performance issues with a single pass approach to packet processing that it claims is unique in the industry. By performing operations once per packet, the single pass software seeks to eliminate many redundant functions.

As a packet is processed, networking is performed once, policy lookup is performed once, application identification and decoding are performed once, and signature matching for any and all threats and content is performed once. The "one-pass" concept significantly reduces the amount of processing overhead required to perform multiple functions in one security device. The single pass software uses a stream-based, uniform signature matching engine for content inspection.

Instead of using separate engines and signature sets (requiring multi-pass scanning) and instead of using file proxies (requiring file download prior to scanning), the single pass architecture scans traffic for all signatures once and in a stream-based fashion to avoid latency introduction. This single pass software is then integrated with the purpose-built platform Palo Alto Networks has developed, using dedicated processors and memory for networking, security, content scanning and management.

PUT TO THE TEST

The Test Bed Configuration

For our test bed, in addition to the Spirent Avalanche NEXT; which is based on an Avalanche C100MP appliance and the Avalanche NEXT Virtual Controller (laptop-based) and the device under test (DUT); the PA-5020, we added a D-Link DGS-3420 Ethernet switch with a combination of Gigabit and

10Gigabit ports in order to connect our Avalanche C100MP appliance and the DUT.

We connected four 10Gigabit interfaces (SFP+) from the Avalanche appliance into the switch and then fed multiple Gigabit connections to the PA-5020 via four VLANs, as we wanted to focus on the real-world, threat prevention capabilities of the firewall—the claimed 2Gbps throughput limit.

We set up initially a basic "allow-allow" rule on the firewall, in order to guarantee that traffic was passing through the device as planned with no unintended blocking and attempted to achieve the stated throughput claims of Palo Alto Networks. Note that by default, the PA-5020 is always running in App-ID enabled mode, but we found in pre-testing that, regardless of which additional features where enabled, performance remained constant (i.e., there was no drop in throughput when additional features were enabled, so all test results featured here are with all firewall features ON) that is firewall, plus IPS, plusAnti-virus, plus Anti-Spyware. We did also retest with a far more extensive firewall rule set, 100 rules, to compare performance and device utilisation. Note: all performance graphics are taken directly from the Avalanche NEXT reports.

Performance Testing

Gigabit Throughput—Mixed Applications

With the PA-5020 in all ON mode and a basic firewall rule set, we initially ran tests at Gigabit speeds to ensure consistency and the basic ability to fill a Gigabit pipe.

(15)

In line with our requirement to go for real-world performance testing, we used Avalanche NEXT’s "Throughput with mixed applications" test option where you can mix and match applications and protocols in order to model any real-world network behaviour. Exactly what percentage of what application/protocol is part of the mix can be defined by the user. We started with a pre-defined mix based on monitoring several user behaviour examples.

Figure 6 – Gigabit Test: App Mix Plus Successful Transactions

We achieved Gigabit performance rates without problems: the 0.03% failed transactions can be due to simply connections not having time to close at the end of the test, so this is rounded up to a 100% pass rate by Avalanche NEXT. Even in all ON mode, the PA-5020 utilisation was mainly below 30%, occasionally peaking at 35% on these tests.

(16)

Two Gigabit Throughput—Mixed Applications

The required throughput rate was set to 2Gbps and the tests were rerun.

Figure 8 – Two Gigabit Test: App Mix Throughput Achieved

Again, we achieved the required throughput rate and were able to sustain 2Gbps without problems. Utilisation on the PA-5020 again peaked around 35%.

(17)

Two Gigabit Throughput—Mixed Applications with 100 Firewall Rules

The previous test was then repeated, but with a far more complex firewall rule set enabled; 100 different rules applied to every incoming data packet during the testing. This was perceived to put significantly greater stress on the PA-5020 processing capabilities.

Figure 9 – Two Gigabit Test: App Mix With 100 Firewall Rule Enabled

Once more we were able to achieve desired throughput rate, sustained and without problems, despite having a very complex set of firewall rules to now work through. Utilisation on the PA-5020 did increase significantly; some CPU cores were peaking at around 90% during the tests, but there was still plenty of system overhead and very little memory usage.

(18)

2.5Gbps Test - HTTP 1.1

Since we effectively had 2x2Gbps connections in our test configuration, we decided to double check the limits of the PA-5020, by running a test consisting of almost entirely HTTP 1.1 traffic, the bar set to 2.5Gbps, all other configuration details as on the previous test and with the 100 rule firewall configuration in place. As expected we achieved 2Gbps maximum, with PA-5020 utilisation as in the previous test.

(19)

SUMMARY & CONCLUSIONS

Our challenge was –to use the Spirent Avalanche NEXT test product to gauge whether high-level performance testing has to be complex or, despite the actual complexity of the testing beneath, can the "human" mechanics of the testing actually be simplified?

In order to put the theory to the test, we worked with Palo Alto Networks, who provided an

Enterprise-level Next Generation Firewall (NGFW) in the form of the PA-5020 to enable us to carry out tests with 10GbE and Gigabit links. We set up a test bed and ran a series of tests aimed at confirming Palo Alto Networks "marketing figures" for the PA-5020; the ability to run at 2Gbps with every firewall feature enabled, regardless of traffic type.

From a performance perspective, we found that the PA-5020 lived up to its billing. The marketing figures were achieved in each test and we found that adding features one by one (IPS, Anti-Virus, Anti-Spyware) had zero impact on performance. System utilisation did increase but there was always some spare headroom, even when theoretically maxed out.

From the Spirent Avalanche NEXT perspective, we found the tests quick and easy to configure and rerun while making as many changes as we wished to elements such as the application/protocol mix. We were also able to make extensive use of the result/statistics provided by the auto-generated reports, examples of which you can see in this document.

Overall, we not only proved that the Palo Alto Networks PA-5020 performs as advertised, but we also proved that this level of testing need not be complex to setup and administer.

Can Security Testing Be Simple?