-Comparison of Next
Generation Firewall
offerings from Cisco,
Intel Security and
Palo Alto
1. Introduction
Next generation firewalls, commonly abbreviated as NGFW, build on the capa-bilities of traditional stateful firewalls by adding application awareness and deep packet inspection capabilities to detect and block threats on the network. Tra-ditional stateful inspection firewalls have essentially become obsolete because they do not inspect the payload of the packet and have no application aware-ness to distinguish between legitimate busiaware-ness application traffic and that of a malicious attack. In contrast, instead of allowing all traffic on typical Web ports, an NGFW can distinguish between specific applications (for instance, Netflix vs. Salesforce.com) and then apply policies based on business rules.
Gartner defines an NGFW as “a wire-speed integrated network platform that performs deep inspection of traffic and blocking of attacks.” At minimum, Gartner states, an NGFW should provide:
• Application awareness, full stack visibility and granular control • Non-disruptive in-line bump-in-the-wire configuration
• Standard first-generation firewall capabilities, such as network-address translation (NAT), stateful protocol inspection (SPI), and virtual private networking (VPN)
• Integrated signature-based Intrusion Prevention System (IPS) engine
• Ability to incorporate information from outside the firewall, such as directory-based policy, blacklists, and white lists
• Upgrade path to include future information feeds and security threats, and Secure Socket Layer (SSL) decryption to enable identifying undesirable encrypted applications
Next Generation
Firewall Capabilities
Assessment
-Application awareness is what makes a firewall a next
generation firewall. NGFW vendors use a variety of techniques, including predefined application signatures, header inspection, and payload analysis to determine specific applications. The NGFW stores a library of approved applications and allows them to traverse the network, while examining the data packets for any anomalies. Along with predefined applications, NGFWs can also “learn” new applications by watching how the applications behave. The NGFW creates a baseline of normal behaviors and can alert administrators if the application deviates from normal.
This study assesses the capabilities of three NGFW vendors:
1. Cisco Adaptive Security Appliance (ASA) with FirePOWER services
2. Intel Security McAfee NGFW 3. Palo Alto NGFW
Our study evaluates the vendors’ NGFW capabilities, including strengths and weaknesses, based on the technical assessments, testing, and insights provided by Miercom, NSS Labs, Gartner, ESG Labs, and vendor published specifications. Our goal is to provide federal agencies with market intelligence to help them acquire an NGFW that most closely aligns with their requirements and mission. This white paper is not meant to be a detailed engineering report, but rather a concise summary of the capabilities provided by Cisco, Intel Security, and Palo Alto. For a deeper understanding of the NGFW capabilities,
evaluation instrumentation, and test methodologies, the reader is referred to the original sources from Miercom, NSS Labs, Gartner, ESG Labs, and vendor published specifications. A complete list of references is provided at the end of this white paper.
2. Next Generation Firewall
Vendor Overview
2.1 Cisco ASA with FirePOWER Services
Although known for its routers and switches, Cisco also has a strong security focus and provides security products that are used by enterprises and data centers. Cisco’s first move into the NGFW market began with the conversion of the legacy ASA firewall into an NGFW product called ASA CX. Unfortunately, Cisco ASA CX proved to be a limited product that never gained wide acceptance in the market. To strengthen its NGFW capabilities, Cisco acquired Sourcefire. Sourcefire provided Cisco with the intellectual property to deliver a next generation firewall and IPS. However, Cisco uses a Sourcefire blade (rebranded as FirePOWER) in the same ASA chassis as its legacy ASA firewall. This means that the end product is inheriting the limitations of the chassis. Ultimately, the NGFW capability in Cisco ASA is a “bolt-on” solution.
Strengths • Cisco brand
• High effectiveness against signature-based threats • Support for high availability (HA) failover in active/
standby mode Weaknesses
• Weak NGFW capabilities: cannot run IPS and application control simultaneously. If you can’t run IPS and
application control simultaneously, then it does not really qualify as NGFW
• No support for clustering, active/active load balancing capability
• Active/standby capabilities are limited and result in feature loss, including advanced threat detection in ASA 9.x code
Why these vendors?
SwishData selected these vendors because
they tend to dominate our discussions with
customers. Cisco is the major network vendor
and usually the default firewall choice for
many organizations simply because it is Cisco.
The other two NGFW vendors, Intel Security
and Palo Alto, were shown in Department of
Defense (DoD) testing to be the only NGFW
-• FirePOWER management console can only support up to
150 devices. Once 150 devices are exceeded, customers must purchase another console. Management of ASA legacy features requires secondary management through Cisco Security Manager (CSM).
• Poor performance against Advanced Evasion Techniques (AETs)
2.2 Intel Security / McAfee NGFW
McAfee is a wholly owned subsidiary of Intel and has undergone rebranding from McAfee to Intel Security. However, to retain McAfee brand cachet, most products within Intel Security’s portfolio retain the McAfee name (e.g., McAfee NGFW). Intel Security’s NGFW offering stands out from its competitors by leading the security market in the field of AET research, which is critical for being able to detect advanced persistent threats (APTs) in an enterprise network. Gartner identifies McAfee NGFW as a “visionary product” in the Gartner Magic Quadrant, because it has firewall features that are not seen in competitor’s offerings. The McAfee NGFW can be purchased as a hardware appliance, a virtual machine, and MIL-STD-810 ruggedized tactical appliances.
Strengths
• Market leader in AET detection and remediation • Highest throughput of any NGFW in the market with all
security features enabled
• Designed to provide ASIC-like performance in x86 architecture
• ASIC-like performance of virtual appliances running in VMware environment
• Built-in active/active clustering that scales to 16 nodes, with dynamic load balancing. No scheduled downtime required for software upgrades within a cluster.
• Integrates with McAfee’s ecosystem of security products, including Host Based Security System (HBSS) and Global Threat Intelligence (GTI)
• McAfee Security Management Center (SMC) supports up to 2000 managed devices
• Management center can receive logs from other platforms, allowing SMC to act as a log server
• Low total cost of ownership (TCO), as recognized by the NSS Lab Security Value Map (SVM)
Weaknesses
• Poor US presence and install base. Few US customers available as reference.
• Unique user interface (UI) means that due to the learning curve, the end user may require more up-front training
• No on-board management. NGFW appliances need to be deployed together with SMC server for integrated management. On-board management capability is currently being added for inclusion with the next firmware release.
2.3 Palo Alto NGFW
Palo Alto Networks is a pure-play network security company. Gartner assesses Palo Alto as a leader, largely because of its NGFW design, consistent displacement of competitors, rapidly increasing revenue and market share, and market disruption that forces competitors in all quadrants to react. However, Gartner does not test the products and so is unable to discuss the limitations of the Palo Alto NGFW. For example, Palo Alto struggles with performance when additional features are turned on and requires third-party software to support clustering, which limits its scalability. Palo Alto has achieved market success because it was the first vendor to offer a firewall with true NGFW capabilities: firewall, IPS, DPI, application control, user ID visibility, and anti-malware.
Strengths
• Robust application control and DPI capabilities • Strong IPS solution with the NGFW
• ASIC-based, optimized data path allows for high throughput performance
• Strong central management and reporting capabilities for smaller deployments through Pal Alto’s Panorama management console
• Integration with Palo Alto’s WildFire, which is a sandbox solution performing run-time code analysis of a suspect file
-Weaknesses
• “Caution” rating was issued by NSS Labs because products running PAN-OS v6.0.3 are susceptible to severe evasion failures, which cannot be publicly disclosed without putting Palo Alto Networks customers at risk, since there are currently no known workarounds without upgraded to a newer version PAN-OS. This may also affect other versions released after the last known good version tested by NSS, PAN-OS v4.1.9.
• Performance declines below advertised throughput as additional capabilities on the firewall are enabled. Effectively, customers are forced to turn off some of the NGFW capabilities if they would like to retain high network traffic throughput on the device.
• Requires a 3rd party load balancer solution to perform clustering above 2 nodes.
• Unreasonably high TCO, as assessed by NSS Labs, which placed Palo Alto in the lower left-hand corner of NSS Labs SVM.
• Hardware ASIC performance does not translate into virtualized environment .
3 Next Generation Firewall
Comparison Matrix
Based on the research data from Miercom, NSS Labs, Gartner, and ESG Labs, we compiled a list of NGFW capability parameters and put them into a matrix for comparing NGFW products. Each capability was given a grade 1 through 5 as follows:
5 - Excellent — Capability is better than that offered by most competing products on the market
4 - Good — Capability is robust, but may present a few non-critical shortcomings
3 - Fair — Capability is adequate, but there are better products out there
2 - Behind the Competition — Capability competes poorly with that offered in other products
We then rated each of the three NGFW products in 10 important capability areas. The results are shown in the NGFW comparison matrix on the next page.
The McAfee NGFW was clearly superior to the Cisco ASA with Firepower and the Palo Alto NGFW, scoring 46 out of 50 possible points. In every category, the McAfee NGFW received a rating that was either higher than or equal to the other NGFWs.
The discussion below describes each of the capability parameters used in the NGFW comparison matrix and explains how we assigned our ratings.
Application Visibility
Application visibility is the core NGFW capability. Different vendors use different techniques to identify applications within network traffic. Some vendors use basic techniques such as hash, string, and URL matching, while others employ sophisticated application fingerprinting method-ologies.
Within DoD, an agency conducted a number of tests to determine which NGFW products performed best in the area of application identification and categorization. The agency used Ixia XM12 and BreakingPoint (now acquired by Ixia) FireStorm network test appliances to generate application traffic and let the NGFW products identify the applications on the wire. Only Palo Alto and McAfee NGFWs were able to successfully identify all applications. In short, only Palo Alto and Intel Security have the “special sauce” to accurately do application fingerprinting within a firewall. For a copy of the report, please contact your Chief Information Security Officer (CISO). SwishData can help direct you to the right information source.
Signature-based Threat Detection
Signature-based threat detection is the basic capability of all modern firewalls, not just NGFWs. The signature-based threat detection performance depends on how quickly the firewall signatures are updated after signatures for new threats emerge. One could argue that because McAfee has its Global Threat Intelligence (GTI) worldwide feed, McAfee NGFW would be updated more quickly than offerings from Cisco or Palo Alto. However, based on our research, we
-Web Security Effectiveness
According to Miercom, web security effectiveness cov-ers protection against drive-by-installcov-ers, complex web exploits, phishing, and malicious redirects. Cisco offers a web security in a form of its IronPort web security appli-ance. However, some of the functionality is included in the ASA firewall with FirePOWER services. According to NSS Labs, the web capability is very good, which is why we opted for the 4 rating. Referencing Miercom web security tests, McAfee was a capable performer. However, most surprising was Palo Alto’s poor URL filtering functionality, which is available via subscription. Palo Alto only yielded 3 percent block rate in Miercom’s web security effectiveness test.
Dangerous Website Filtering
Dangerous website filtering refers to the security device’s ability to detect and block various types of risky web content, such as sexual material, gambling, proxy avoidance, and hacking. Blocking these types of web content is an important aspect of controlling online access to minimize loss of user productivity, manage bandwidth costs, prevent potentially malicious content from entering the enterprise network and meet compliance requirements. Our grading was based on reports from Cisco ASA NSS Labs reports and Miercom web security testing.
AET Detection
AET detection is major factor for organizations concerned with APTs and zero-day exploits. The pioneering vendor that began implementing AET detection methods within a firewall platform was Stonesoft. As Stonesoft NGFWs gained popularity, Intel Security acquired Stonesoft to compete with Palo Alto in the NGFW market.
NGFW Capability Cisco ASA
w/ FIREPOWER McAfee NGFW Palo Alto NGFW
Application Visibility 3 5 5
Signature-based Threat Detection 5 5 5
Web Security Effectiveness 4 4 2
Dangerous Website Filtering 4 4 3
AET Detection 3 5 3
Throughput 2 5 4
Scalability 2 5 3
High Availability 2 5 3
Management & Reporting 3 4 4
TCO 2 4 1
Total Score: 30 / 50 46 / 50 33 / 50
Application awareness is
not all the same.
With NGFW being the new big buzzword and
every vendor wanting to jump on the NGFW
bandwagon, many vendors have resorted
to shortcuts. Some call their latest firewall
offering an NGFW and claim that it does
application awareness, when it only performs
basic application categorization, if anything
at all. If an unsuspecting customer were to
procure this NGFW-labeled product, he or she
would find NGFW capabilities to be woefully
inadequate.
-While testing evasions at different layers of the network,
Intel Security began to learn about more complex and dynamic evasions appearing in the wild. In 2010, Intel Security published a report on the discovery of AETs, and highlighted the vulnerabilities of most security devices at the time. Intel Security asserts that most security devices are still vulnerable to AETs today. Intel Security runs millions of evasion combinations in its labs daily, and shares its findings with the Computer Emergency Readiness Team (CERT) and numerous security vendors. The Evader tool was developed to provide in-house testing capability for companies that deploy network security de-vices using deep packet inspection, such as IPS and NGFW. Companies can use Evader for real-world tests of their protection against AETs, thus enabling them to improve security levels and evaluate the results against vendor claims and published lab results. Evader is provided free of charge by Intel Security at http://evader.mcafee.com/. It is important to note that Evader is not a hacking tool or a penetration test harness. Evader simply tests if a known exploit can be delivered using AETs through currently installed security devices to a target host.
When it comes to AET detection, McAfee NGFW is an undisputed leader with Cisco and Palo Alto trailing behind. The comparison matrix AET grades reflect this.
Throughput
Palo Alto claims it is the only vendor in the industry with an optimized data plane because of the proprietary ASICs used for wire-speed processing. However, this is only true in limited situations. From field experience, Palo Alto shows a steep decline in throughput performance as features are turned on. Therefore, to get the advertised performance numbers, many of the NGFW features need to be disabled, thereby lowering the security posture of the product.
Cisco ASA with FirePOWER services experiences similar issues. Cisco has had backplane throughput limitations starting with its Catalyst switches; consequently, Cisco acquired Nuova Systems in 2008 to get the technology for Cisco Nexus switches. The same largely holds true with the updated ASA firewalls. They are low throughput, only going to 10 Gbps when application control and IPS
and McAfee NGFW can do a whopping 120 Gbps with all features turned on.
Scalability & High Availability
When talking scalability, Cisco ASA does not do well with large environments. However, if the need is for a small business or a branch office, Cisco ASA could be completely adequate. Cisco also does not do clustering. Its firewalls operate in active/standby failover mode only. Palo Alto does well for small to mid-size environments. Palo Alto NGFWs can work in pairs to form a single NGFW cluster. However, beyond that, one needs to use a third party load balancer to scale the NGFW deployment.
McAfee NGFW can work in clusters of 16 nodes with terabit throughputs. McAfee NGFW also offers the ability to do capacity and software updates to the cluster without any disruption. It is the only vendor with that capability to date. Overall, the McAfee NGFW product does well for deployments small, mid-sized, and large. Moreover, in an effort to gain greater market share from Palo Alto, McAfee NGFW products are competitively priced.
Management & Reporting
Per Network World magazine’s June 13, 2013 issue, Cisco still has significant work to do in improving the manage-ment, integration, threat mitigation and application con-trols. Palo Alto has its Panorama management console. Panorama provides the ability to manage a distributed network of firewalls from a centralized location. Us-ing Panorama, one can view firewall traffic, manage all aspects of device configuration, push global policies, and generate reports on traffic patterns or security incidents. The issue with Panorama is that it does not scale in large networks. For large deployments, multiple Panorama ap-pliances are required.
McAfee NGFW offers a similar management solution called the Security Management Center (SMC). The SMC al-lows one to manage, monitor, log, and report on most In-tel Security/McAfee products from one console. Addition-ally, the SMC can manage third-party switches, routers, and security appliances, and act as an external log server for other devices to send logs to. You can efficiently auto-mate routine tasks, reuse elements and utilize numerous
-Total Cost of Ownership
The best way to discuss the TCO aspect of NGFW products is to examine the NSS Labs Security Value Map (SVM), which breaks down NGFW cost into “TCO by protected-Mbps.”
4 Solution Review and
Recommendation
NGFWs combine application awareness and deep packet inspection to give organizations more control over applications while also detecting and blocking malicious threats. In the past several years, it seems as if every vendor has begun offering an NGFW solution. However, as we have seen, only two vendors have a robust application visibility function that stands up to scrutiny: Palo Alto and Intel Security/McAfee. The other vendors may detect only some application traffic, while the rest will go uncategorized.
Larger organizations need to be increasingly concerned with the advent of APTs and the risk they pose. The majority of APTs are delivered through covert channels by means of advanced evasion. This makes AET
capabilities critical for any NGFW product considered by an organization. All NGFW products on the market tout evasion capabilities. However, as can be proven by the Evader tool, most fall short detecting even the basic of evasions. Intel Security is the undisputed leader when it comes to AETs. Palo Alto is catching on as well. In October
2014, Palo Alto delivered a “silent update” in its PAN-OS v6.05-h3 code that fixed major evasion holes published by the NSS Labs. Cisco’s evasion capabilities are still weak. Scalability and high availability are two other points to consider. If your organization does not anticipate growth, scalability may not be an issue. If your organization does not have a high throughput requirement, you may be fine with your firewall having just an active/standby HA mode. However, if you cannot tolerate any downtime, a more appropriate solution may be the one that can support clustering capabilities and hitless upgrades, all while maintaining high throughput.
To conclude with some recommendations, for small to mid-size organizations that would like to use Cisco because they are heavily invested in Cisco products, the Cisco ASA with FirePOWER services may be an adequate choice. However, your organization will be missing important security capabilities, and so will have to purchase a separate security appliance (e.g., an IPS/ IDS) to augment deficiencies in the ASA. Palo Alto is a good product that does very well in all but very large deployments. However, Palo Alto NGFW is incredibly costly and, as we have seen, does surprisingly poorly in the web-filtering category. Standing tall in our evaluation is Intel Security’s McAfee NGFW. It exceeds Palo Alto in seven of the ten capability categories, including throughput, HA, and AET detection. McAfee NGFW equals Palo Alto in the other three categories. And because Intel Security is trying to recapture market share from Palo Alto, McAfee NGFW pricing is very competitive.
17 Feagles Road • Warwick, New York 10990 (703) 531-8526 Phone • (703) 852-7904 Fax [email protected]
About SwishData
www.swishdata.com
We’re the cybersecurity and data performance architects. SwishData ensures the performance, affordability, and security of your agency’s data infrastructure through both architecture and deployment.
