Best Practices Guide Revision C. McAfee Network Security Platform 8.3

(1)

Best Practices Guide

Revision C

(2)

COPYRIGHT

TRADEMARK ATTRIBUTIONS

Intel and the Intel logo are registered trademarks of the Intel Corporation in the US and/or other countries. McAfee and the McAfee logo, McAfee Active Protection, McAfee DeepSAFE, ePolicy Orchestrator, McAfee ePO, McAfee EMM, McAfee Evader, Foundscore, Foundstone, Global Threat Intelligence, McAfee LiveSafe, Policy Lab, McAfee QuickClean, Safe Eyes, McAfee SECURE, McAfee Shredder, SiteAdvisor, McAfee Stinger, McAfee TechMaster, McAfee Total Protection, TrustedSource, VirusScan are registered trademarks or trademarks of McAfee, Inc. or its subsidiaries in the US and other countries. Other marks and brands may be claimed as the property of others.

LICENSE INFORMATION License Agreement

NOTICE TO ALL USERS: CAREFULLY READ THE APPROPRIATE LEGAL AGREEMENT CORRESPONDING TO THE LICENSE YOU PURCHASED, WHICH SETS FORTH THE GENERAL TERMS AND CONDITIONS FOR THE USE OF THE LICENSED SOFTWARE. IF YOU DO NOT KNOW WHICH TYPE OF LICENSE YOU HAVE ACQUIRED, PLEASE CONSULT THE SALES AND OTHER RELATED LICENSE GRANT OR PURCHASE ORDER DOCUMENTS THAT ACCOMPANY YOUR SOFTWARE PACKAGING OR THAT YOU HAVE RECEIVED SEPARATELY AS PART OF THE PURCHASE (AS A BOOKLET, A FILE ON THE PRODUCT CD, OR A FILE AVAILABLE ON THE WEBSITE FROM WHICH YOU DOWNLOADED THE SOFTWARE PACKAGE). IF YOU DO NOT AGREE TO ALL OF THE TERMS SET FORTH IN THE AGREEMENT, DO NOT INSTALL THE SOFTWARE. IF APPLICABLE, YOU MAY RETURN THE PRODUCT TO MCAFEE OR THE PLACE OF PURCHASE FOR A FULL REFUND.

(3)

Preface

This guide provides the information you need to configure, use, and maintain your McAfee product.

Contents

About this guide

Find product documentation

About this guide

This information describes the guide's target audience, the typographical conventions and icons used in this guide, and how the guide is organized.

Audience

McAfee documentation is carefully researched and written for the target audience. The information in this guide is intended primarily for:

• Administrators — People who implement and enforce the company's security program.

• Users — People who use the computer where the software is running and can access some or all of its features.

Conventions

This guide uses these typographical conventions and icons.

Italic Title of a book, chapter, or topic; a new term; emphasis Bold Text that is emphasized

Monospace Commands and other text that the user types; a code sample; a displayed message Narrow Bold Words from the product interface like options, menus, buttons, and dialog boxes Hypertext blue A link to a topic or to an external website

Note: Extra information to emphasize a point, remind the reader of something, or provide an alternative method

Tip: Best practice information

Caution: Important advice to protect your computer system, software installation, network, business, or data

(6)

Find product documentation

On the ServicePortal, you can find information about a released product, including product documentation, technical articles, and more.

Task

1 Go to the ServicePortal at https://support.mcafee.com and click the Knowledge Center tab. 2 In the Knowledge Base pane under Content Source, click Product Documentation.

3 Select a product and version, then click Search to display a list of documents. Preface

(7)

1

Introduction

McAfee®_{Network Security Platform [formerly McAfee}®_IntruShield®_{] is a combination of network}

appliances and software, built for the accurate detection and prevention of intrusions and network misuse.

We recommend that you follow some of the best techniques and tips to use McAfee Network Security Platform most effectively. This can save considerable time during the installation and tuning process of the system.

Following chapters outline the best practices for Network Security Platform.

Pre-installation checklist

There are some important tasks that you should consider before McAfee®_{Network Security Manager}

[formerly McAfee®_IntruShield®_{Security Manager] software installation. For more information, see}

Planning for installation, McAfee Network Security Platform Troubleshooting Guide.

1

(8)

1

Introduction

(9)

2

Cabling best practices

It is a common practice to physically cable the monitoring ports, only after the McAfee®_Network

Security Sensor (Sensor) has been fully configured.

In other words, most administrators cable the console and management ports, use those connections to configure the solution, and only physically introduce the Sensor into the scanning process once the proper scanning policies are in place, the monitoring ports have been configured, the latest signature set has been downloaded, and so on.

Also, in the most security-conscious environments, or those with congestion problems, a private network is often used to connect the Sensor management ports to the McAfee®_{Network Security}

Manager (Manager). This is another best practice in terms of cabling the Sensors.

2

(10)

(11)

3

Hardening the Manager Server for

Windows platform

Implementation of Manager varies from environment to environment. The Manager's physical and logical position in the network influences specific remote access and firewall configuration

requirements. The following best practices on managing configurable features on Manager impacts the security of Manager.

These steps are applicable to Windows Server 2008 and Windows Server 2012 editions.

Contents

Pre-installation Installation Post-installation

Pre-installation

Use a dedicated machine for the Manager server and then install Manager and the embedded MySQL database. Other than the host-based firewall, no other software should be installed on the server. Before installation of Manager do the following:

• Ensure that the server is located in a physically secure environment. • Connect the server on a protected or isolated network.

• If the hard disk is old, use fdisk (a command line utility) to remove all partitions and create new partitions.

Installation

Installation of Manager should be performed as follows: • Install the US version of Windows Server.

• Use NTFS on all partitions.

3

(12)

Post-installation

After installation of Manager perform the following installations:

• Install the latest Windows Server patches, service packs, and hot fixes from Microsoft. • Install a Virus Scanner and update the signatures.

Exclude "McAfee®_{Network Security Manager (Manager)" and "MySQL" directories from being}

scanned.

Also keep a check on the following:

• Minimize the number of Windows roles and features that are installed. • Uninstall applications that are not necessary.

Disable non-required services

Disable the following services: • DHCP Client

• FTP

• Print spooler

• Remote access auto connection manager • Remote procedure call locator

• Remote registry • Server

• TCP/IP NetBIOS helper service • Telephony service.

Enable these services only if it is absolutely required.

Set system policies

Ensure to set the following system policies:

• Implement the System key and strong encryption of the password database by running SYSKEY.EXE

• Use Microsoft security compliance toolkit or set local security policy • Display legal notice at during interactive logon window.

• Do not display username that was earlier used to login. • Disable Posix

• Clear virtual memory page file during shutdown • Disable autorun

• Disable LMHOSTS lookup while setting the advanced TCP/IP settings.

3

Hardening the Manager Server for Windows platform

(13)

Set user policies

Make sure to set the following user policies: • Rename the administrator account. • Disable guest account.

• Passwords should be at least 8 ASCII characters. • Enable locking of screensaver.

Set the desktop firewall

It is recommended that a desktop firewall operates on the Manager server. The following ports are required for Manager-Sensor communication.

Ensure that there are no other open ports using a scanning tool such as McAfee Vulnerability Manager.

Port Description Communication

80 HTTP port Client to Manager

443 HTTPS Client to Manager

3306 MySQL database Open only while using external SQL database 8500 Command channel(UDP) Manager to Sensor

8501 Install channel (TCP) (1024-bit) Sensor to Manager 8502 Alert channel (TCP) (1024-bit) Sensor to Manager 8503 Packet log channel (TCP) (1024-bit) Sensor to Manager 8504 File transfer channel (TCP) Sensor to Manager 8506 Install channel (TCP) (2048-bit) Sensor to Manager 8507 Alert channel (TCP) (2048-bit) Sensor to Manager 8508 Packet log channel (TCP) (2048-bit) Sensor to Manager 8509 Bulk file transfer channel for 2048-bit

certificates. Sensor to Manager

8510 Bulk file transfer channel for 1024-bit

certificates. Sensor to Manager

8555 Alert viewer (TC) Client to Manager

When email notification or SNMP forwarding is configured on Manager and there is firewall between Manager and SNMP Server, ensure that the following ports are allowed through firewall.

Port Description Communication

25 SMTP port Manager to SMTP server

162 SNMP forwarding Manager to SNMP server

If you have McAfee ePO™_{integration configured on Manager, and there is firewall between Manager}

and the McAfee ePO™_{Server, ensure the following port is also allowed through firewall.}

Port Description Communication

8443 McAfee ePO™_{communication port} _{Manager to McAfee ePO}™_server

(14)

Configure audit events

Set the following events to be audited:

• Audit account logon events • Audit policy change (Success) • Audit account management • Audit privilege use (Failure) • Audit logon events • Audit system events (Success) • Audit object access (Failure)

3

(15)

4

Hardening the MySQL installation for

Windows platform

This section describes methods for hardening your MySQL installation.

Contents

Harden the MySQL installation

Other best practices for securing Manager

Harden the MySQL installation

Ensure the cmd window used for making changes to database tables in the "mysql" database stays opened in the mysql shell until validation is completed.

This is necessary to enable you to rollback the changes in case you need to. Rollback procedures are shown at the end of this section.

Use another cmd window, where necessary, to validate hardening changes you have made.

Remove test database

Remove the 'test" database from the server.

1. Start My SQL. mysql> use mysql;

2. Backup db table to do dbbackup before changing it. mysql> create table db_backup as select * from db;

3. Validate that the backup table was created and row count

matches that of the mysql.db table. mysql> select count(*) fromdb_backup; 4. Check all the databases on the Manager server. mysql> show databases; 5. Remove the test db. mysql> drop database test; 6. The test database should no longer be listed. mysql> show databases;

Remove local anonymous users

To remove local anonymous users:

1. Look for blank entries for user. mysql> select host,db,user from db;

2. Remove anonymous access to databases mysql> update db set host="localhost" where user="";

(16)

3. Remove anonymous/blank accounts mysql> flush privileges; 4. Validate that "localhost" replaced % entry under the host

column. You will also notice you will now need to qualify username and password on the local machine to get into mysql shell from the mysql.exe CLI.

Remove remote anonymous users

To remove remote anonymous users, you harden mysql.exe CLI access by forcing the requirement for a username and password to get into the mysql shell as follows.

Start MySQL. mysql> use mysql;

Back up the user table to user_backup before

changing it. mysql> create table user_backup asselect * from user; Validate that the backup table was created and row

count matches that of the mysql.db table. mysql> select count(*) fromuser_backup;

List all users and hosts. mysql> select user,host from user; Remove anonymous/blank accounts. mysql> delete from user where user=""; Validate that rows with blank user columns have been

removed. mysql> select user,host from user;

Secure MySQL remote access

This section provides two options for removing remote access. • Remove individual users' remote access

• Remove ALL remote access (Recommended)

Removal of individual user's remote access

Do ONE of the following:

• Remove admin (Network Security Platform user) remote access

mysql> delete from user where host!='localhost' and user='admin';

(The admin user cannot login remotely; however Manager root can. Use second cmd window to validate.)

mysql>flush privileges;

• Remove root remote access (Recommended minimum action)

mysql> delete from user where host!='localhost' and user='root';

This ensures that the root user cannot login remotely; however Manager user can log in remotely. Use second cmd window to validate.

mysql>flush privileges;

Remove ALL remote access

mysql> delete from user where host!='localhost'.

ALL user access is disabled including Manager users from remote host(s).

4

Hardening the MySQL installation for Windows platform

(17)

Use another cmd window to validate; you can ONLY log in to the MySQL CLI on the Manager server by qualifying username, password and db. For example: mysql -uadmin -pXXX lf.

How to roll back your changes

If you need to roll back your changes, use the following commands:

To roll back changes made to the mysql.db table from the mysql.db_backup table: mysql> rename table db to db_1;

mysql> alter table db_backup engine=MyISAM; mysql> rename table db_backup to db;

mysql> flush privileges;

To roll back changes made to the "mysql.user" table from mysql.user_backup table: mysql> rename table user to user_1

mysql> rename table user_backup to user; mysql> flush privileges;

Removal of debug shell at port 9001

In addition to denying traffic over port 9001 and 9002 (as per Install a desktop firewall), the debugging shell that runs on port 9001 can be disabled by modifying the value of the

iv.policymgmt.RuleEngine.BSH_Diagnostics_Port record in the iv_emspropertiestable. To disable the port, set the value in the field called "value" = -1

(18)

Other best practices for securing Manager

• Use a clean, dedicated machine for the Manager server and perform a fresh install of the Manager software, including the installation of the embedded MySQL database. No other software should be installed on the server, with the exception of a host-based firewall as described above.

• Make sure the PC is in an isolated, physically secure environment

• Disallow access to the installation directory and its sub-directories to anyone other than authorized administrators. Use Microsoft Knowledge Base article # 324067 to accomplish this procedure. Disallow the following permissions:

• Read • Modify

• Write • List folder contents

• Read and Write • Full control

4

(19)

5

Large Sensor deployments

When you consider large McAfee®_{Network Security Sensor (Sensor) deployments, (where the number}

of Sensors deployed range from 36 to 100) there are some important tasks which should be considered, before deployment.

McAfee recommends that you have a good understanding on the best techniques required to accomplish these tasks in your deployment scenario, prior to the deployment.

• Concurrent Signature Set and Sensor Software downloads — In 6.0.7.x and above, the Manager provides an option for parallel processing of Sensor software and signature set updates. In earlier releases of 6.0, when multiple Sensors are configured to your Manager, any software update on the Sensors had to be performed individually. If you are using 5.1, then note that this option is available on Manager version 5.1.17.2 and above.

This enhancement is applicable only for Sensor updates in the parent domain. The Sensor updates in the child admin domain is performed in the same method as the earlier releases.

• Sensor Software Updates— All Sensor software updates do require a reboot. A reboot can take up to 5 minutes. You can schedule this process though you can't reboot the Sensor automatically. But any update from the Manager Server causes the process to take place sequentially, one Sensor at-a-time. You can instead use the TFTP method for updating the Sensor image, which helps you to load concurrent images on the Sensor via the Sensor's CLI, at a much faster rate.

For more information, see Upgrading Sensor software via a TFTP server, McAfee Network Security

Platform CLI Guide.

• Central Manager deployment — If you have a large Sensor deployment of 200 Sensors for example, which are deployed across various geographic locations, then consider using a Central Manager at your organization's headquarters and deploy a dedicated Manager for each region. Each Manager will then handle the daily device operations for all Sensors configured to it. Note that when you use a Central Manager, your regional/local Managers can add their own region-specific rules, but cannot modify any configuration established by the Central Manager. Configuration updates to the Sensors must be applied through the local Managers. See McAfee Network Security

Platform Manager Administration Guide for details.

• Usability — Depending on the number of VIDS and Admin Domains defined in your deployment, the Manager Resource Tree can become very crowded, which makes it difficult to locate the resource you require at any point of time. It can also lead to confusion if you have not provided unique, recognizable names for your Sensors and any VIDS you create. Note that the resource names appear both in the Resource Tree of the Manager as well as in Alert data and Reports. Your VIDS names should also be clear and easy for everyone maintaining the network to recognize at a glance. For example, compare a worldwide deployment where Sensors are named "4010-1" through "4010-25" as opposed to "UK-London-sens1," "India-Bangalore-sens1," and so on. • Alert Traffic — Take note of the volume of alerting in your Sensors. Depending on the policies

deployed on your system, there is potential to starve Manager resources since the resulting alerts are passed to the Manager. As the volume of alerting increases, more data is passed into the Manager. The Manager can handle short bursts of high alert volume but on an average, the Manager can handle a total of 1500 alerts per minute from all the Sensors configured to it.

5

(20)

• Start-up load on the Manager— When the Manager starts, establishing connections with all Sensors can be time consuming as Sensors continue to collect alerts. If the communication with the Manager is lost, each Sensor must pass its alert data to the Manager when connectivity is re-established. So, it is required to account for the start-up load on the Manager.

• Concurrent processes— Be aware of the time periods in which your scheduled processes (such as database backup or report generation) occur, and try not to attempt other tasks during that time period, as this can lead to process locking. This includes having many users logged into the system simultaneously.

Contents

Staging Sensors prior to deployment

Recommendations for large Sensor deployment

Staging Sensors prior to deployment

With large or very large deployments, and/or if you are planning to release Sensors to various geographical regions or remote locations, you will have to consider staging your Sensors before you release them to their final destination.

For example, use the McAfee®_{Network Security Manager in a lab environment to push Sensor}

software to the Sensor, make sure that the Sensor is working as expected, and then box the configured Sensor and send it to its final destination. For more information, see Updating the configuration of a Sensor, McAfee Network Security Platform IPS Administration Guide.

Or you might use the TFTP feature to load the Sensor image at one location, before shipping the Sensor to another. For more information, see Upgrading Sensor software via a TFTP server, McAfee

Network Security Platform Installation Guide.

Very large Sensor deployments mean that the number of Sensors deployed is more than 100. Large Sensor deployments have Sensors numbering between 36 and 100+.

Recommendations for large Sensor deployment

Most McAfee Network Security Platform customers begin their deployment in their lab environment. Here they test the Sensor functionality, familiarize themselves with the Manager, and create an initial policy. Once they are comfortable with the product, they deploy the Sensor in a live environment. McAfee provides a few recommendations for this process:

• Spend time creating effective policies before actual deployment. Availability of more information makes the tuning process easier. But policies like the McAfee Network Security Platform provided All-Inclusive policy can overwhelm you with data, if every Sensor in a large deployment is running it without any customization.

• Stagger your Sensor deployment in phases. As each new batch of Sensors provides you with more data points, you can tune your policies more effectively, and become more aggressive in the number of Sensors you deploy in the next phase.

5

Large Sensor deployments

(21)

6

Using active fail-open kits

McAfee supports the following types of passive and active fail-open kits: • 10/100/1000 Gigabit Copper Passive Fail-Open Bypass Kit

• 1 Gigabit Optical Passive Fail-Open Bypass Kit • 10 Gigabit Optical Passive Fail-Open Bypass Kit • 10/100/1000 Copper Active Fail-Open Bypass Kit

• 10/100/1000 Copper Active Fail-Open Bypass Kit with SNMP monitoring • 1 Gigabit Optical Active Fail-Open Bypass Kit

• 10 Gigabit Optical Active Fail-Open Bypass Kit

Fail-open kits can be deployed in production networks for the following reasons:

• Reduce the network downtime to seconds during any Sensor reboot or Sensor failure • Protect your network during link failure on the Sensor

• Bypass the traffic when troubleshooting network issues. This will help you identify or eliminate the Sensor as the cause of network issues

In the passive fail-open kit, if the Sensor goes down, the link has to be renegotiated again between the peer devices and this causes the link to go down for some time. In case of an active fail-open kit, a physical link will be established between the active fail-open kit and the two peer devices even when the Sensor is active. There would not be any link flap even when the Sensor goes down. McAfee recommends deploying active fail-open kits for protection of mission critical networks.

For Virtual IPS Sensors, only 10/100/1000 Copper Active Fail-Open Bypass Kit and 10/100/1000 Copper Active Fail-Open Bypass Kit with SNMP monitoring are supported. For more information, see

Virtual IPS Sensor deployment section in the IPS Administration Guide.

Passive Fail-open

In passive fail-open kits, during normal Sensor in-line, fail-open operation, the Fail-Open Controller or built-in Control port (depending on which controls the Bypass Switch) supplies power and a heartbeat signal to the Bypass Switch.

If this signal is not presented within its programmed interval, the Fail-Open Bypass Switch removes the Sensor from the data path, and moves into bypass mode, providing continuous data flow with little network interruption. While the Sensor is in bypass mode, traffic passes directly through the switch, bypassing the Sensor. When normal Sensor operation resumes, you may or may not need to manually re-enable the monitoring ports from the Manager interface, depending on the activity leading up to the Sensor's failure.

Active Fail-open

(22)

In case of active fail-open kits, during normal Sensor in-line fail-open operation, the built-in monitoring sends a heartbeat signal (1 every second) to the Bypass Switch. If the Sensor does not receive 3 heart beat signals within its programmed interval, the Fail-Open Bypass Switch removes the Sensor from the data path, and moves it into the bypass mode, providing continuous data flow. When the Bypass Switch loses power, traffic continues to flow through the network link, but is no longer routed through the Bypass Switch. This allows network devices to be removed and replaced without network downtime. Once power is restored to the Bypass Switch, network traffic is seamlessly diverted to the monitoring device, allowing it to resume its critical functions.

Considerations

This section discusses the generic requirements and notes that you need to consider with respect to active fail-open kits:

• The currently supported active fail-open kits are not plug and play devices. Initial configuration/ setup is required before you begin.

• The following default options are fixed in McAfee active fail-open kits and cannot be changed: • LFD is set to On

• Bypass Detection is set to Off

Even if you change the configuration for these options using the NetOptics Web Manager or System Manager, the settings of these options on the McAfee active fail-open kit hardware cannot be changed.

• The management port on the active fail-open bypass kits cannot be configured.

• The parameters for the monitoring port must be set to Auto-Negotiate based on the speed, that is, 10/100/1000 Mbps. McAfee recommends that you set the Speed to 100 Mbps full Duplex with Auto-Negotiate enabled to improve performance.

• Unlike passive fail-open kits, an active fail-open kit moves into the bypass mode only when it does not receive the heart beat signals within its programmed interval. When the Sensor monitoring port is manually disabled or the cable is pulled out for example, the Manager displays the port status as AUK (Active Unknown) under Device List / Sensor_Name > Physical Sensor > Port Settings page.

• If you are planning to use the 10/100/1000 copper active fail-open kit with SNMP monitoring, then note that

• Network Security Platform currently supports only SNMP v1 on active fail-open kits.

• You can configure only a single SNMP Manager. The option to configure a secondary SNMP Manager is currently not available.

• The active fail-open kits do not provide any CLI option to view the serial and model numbers of the kits.

• If your network architecture is such that it requires you to remotely manage the active fail-open kits in your deployment, then you can consider one of the following options:

• Use a terminal server to connect to the system console and then connect using a remote login [interoperability issues might be seen while using UPLOGIX Terminal Server]

• Pre-configure the kit with the required settings before shipping.

6

Using active fail-open kits

(23)

7

Effective policy tuning practices

All Network Security Sensors (Sensors) on initial deployment, have the ' Default Prevention' policy loaded on all interfaces. McAfee recommends that you use the Default Prevention policy as a starting point, then customize the policies based on your organization's requirements. The customized policies can be either cloned versions of the default pre-configured policies or custom-built policies that employ custom rule sets. An appropriately tuned policy will reduce false positives.

Though each network environment has unique characteristics, the following best practices can make tuning more efficient and effective.

As you interact with Network Security Platform policies, you encounter the term "attack", not

"signature." Network Security Platform defines an attack as being comprised of one or more signatures, thresholds, anomaly profiles, or correlation rules, where each method is used to detect an attempt to exploit a particular vulnerability in a system. These signatures and checks may contain very specific means for identifying a specific known exploit of the vulnerability, or more generic detection methods that aid in detecting unknown exploits for the vulnerability.

Contents

Analyzing high-volume attacks Managing ignore rules

Learning profiles in DoS attacks

Analyzing high-volume attacks

Take attacks that are generating the most alerts (use the Attack Log page) and investigate their legitimacy. For more information, see McAfee Network Security Platform Manager Administration

Guide.

Many of the top alerts seen on the initial deployment of a Sensor will be common false positives seen in many environments. Typically, at the beginning of the tuning process, it will be evident that your network or security policy will affect the overall level of alerts. If, for instance, AOL IM is allowed traffic on the network, then there might not be a need to alert on AOL IM setup flows.

Managing ignore rules

When a particular alert is declared as a false positive, the next decision is whether to disable the corresponding attack altogether OR apply a particular ignore rule to that attack that will disable alerting for a particular IP address or range of IP addresses. In almost all cases, it is a best practice to implement the latter.

(24)

Consider the McAfee Vulnerability Manager. When McAfee Vulnerability Manager scans a network host, some of this traffic might appear as an attack. However, you are aware of the purpose of this traffic and you do not want the Sensor to take any response action on this traffic. However, if similar traffic is generated by any other server, you want the Sensor to treat it as an attack and respond accordingly. Network Security Platform provides various options to handle such situations.

Every ignore rule created is globally stored, so that the filter can be applied to any Exploit or Reconnaissance attack.

It is also a best practice to document all your tuning activities. The Configuration Report section can be used to assist the documentation process. The Performance Monitoring - Sensor Configuration report will deliver reports that list ignore rules that have been applied and attacks that have been otherwise customized.

For more information, see chapter How to create Ignore rules for an applied IPS policy in the McAfee

Network Security Platform IPS Administration Guide.

Learning profiles in DoS attacks

It is a best practice to let the Sensors learn the profiles of the particular segments they are

monitoring, before tuning DoS attacks. This is Learning Mode operation. The learning process takes two days. During this period it is not unusual to see DoS alerts associated with normal traffic flows (for example, DoS SYN flood alerts reported outbound on a firewall interface to the Internet). After a profile has been learned, the particulars of the profile (number of SYNS, ACKS, and so on) can be viewed per Sensor.

DoS detection can also be implemented using the Threshold Mode. This involves setting thresholds manually for the type of segment characteristics that are learned in Learning Mode. Implementing this mode successfully is critically dependent on detailed knowledge of the segments that the particular Sensors are monitoring.

It is a best practice to have the Sensor re-learn the profile when there is a network change (that is, you move the Sensor from a lab or staging environment to a production environment) or a

configuration change (that is, you change the CIDR block of a sub-interface) that causes a significant sudden traffic change on an interface. If the Sensor does not re-learn the new environment, it may issue false alarms or fail to detect actual attacks during a time period when it is adapting to the new network traffic conditions. There is no need to re-learn a profile when network traffic increases or decreases naturally over time (for example, an e-Commerce site that is getting more and more customers; thus its Web traffic increases in parallel), since the Sensor can automatically adapt to it. For more information, see Managing DoS Learning Mode profiles on a Sensor, McAfee Network Security

Platform IPS Administration Guide.

7

Effective policy tuning practices

(25)

8

Response management

When McAfee®_{Network Security Sensor (Sensor) detects an activity which violates a configured}

security policy, a preset response from the Sensor is integral to the protection or prevention process. Proper configuration of responses is crucial to maintaining effective protection. Critical attacks like buffer overflows and DoS attacks require responses in real time, while scans and probes can be logged and researched to determine compromise potential and the source of the attack.

Developing a system of actions, alerts, and logs based on specific attacks or attack parameters (such as severity) is recommended for effective network security. For example, since McAfee®_Network

Security Platform can be customized to protect any zone in a network, knowing what needs to be protected can help to determine the response type.

If the Sensor is monitoring the network outside of the firewall in inline mode, preventing DoS attacks and attacks against the firewall is crucial. Other suspicious traffic intended for the internal network, such as scans and low-impact well-known exploits, are best logged and analyzed as the impact is not immediate. In this case, a better understanding of the potential attack purpose can be determined. Thus, if you are monitoring outside of a firewall in in-line mode, it is important not to set the policies and responses so fine that they disrupt the flow of traffic and slow down the system.

Remember that response actions are decoupled from alerting. Pay particular attention to this with the Recommended For Blocking (RFB) category of attacks, lest you enable blocking for an attack, but disable alerting, causing the attack to be blocked without your knowledge.

When there are multiple attempts to login to a specific web server from a client, the Sensor detects a reconnaissance Brute force attack (Attack ID 0x40256b00) and raises an alert. Brute force attacks are used by programs, such as password crackers, to try many different passwords in order to guess the correct one. The alerts raised are threshold based. The Sensor may generate an alert even in

scenarios, where a legitimate user keeps on retrying to login to the web server simply because he has forgotten his password. Instances of someone mistyping a password or username on the login are also common. In such cases, valid traffic flow would be blocked or subject to unnecessary responses from the Sensor, leading to a false positive. Consequently, the traffic might be dropped.

When such alerts are seen in high volume, there may be multiple reasons for it, like, a dictionary attack against the web server, or network monitoring systems (like WebSense) not updated with a user password change, and so on.

McAfee®_{Network Security Platform recommends that while configuring a Reconnaissance policy, you}

to edit and set optimum threshold values to suit your particular environment. This avoids unnecessary responses from the Sensor and hindrance to the traffic flow.

For example, if you have a web-server farm behind the Sensor so there are more HTTP logins seen on this segment, in such a scenario you require to set higher thresholds. The default values are good for most environments.

(26)

Sensor response actions

There are multiple Sensor actions that are available for configuration per attack. These include: • Dropping Alert Packets— Only works in in-line mode. Will drop a detected attack packet and all

subsequent packets in the same flow.

• Quarantine— Sensor will quarantine or remediate a host as per the configurations in McAfee®_Network

Security Manager and the Sensor monitoring ports. Quarantine can be enabled per attack in the Policy Editors.

For more information, see McAfee Network Security Platform IPS Administration Guide.

8

Response management

(27)

9

How to create rule sets

A rule set is configured based on attack category, operating system, protocol, application, severity, and benign trigger probability options. Each rule in a set is either an include rule or an exclude rule. An include rule (which should always start a rule set) is a set of parameters that encompass a broad range of well-known attacks for detection. An exclude rule removes elements from the include rule in order to focus the policy's rule set.

Proper creation of rule sets is essential for eliminating false positives and ensuring maximum protection on your network. These best practices can assist while creating rules sets in the McAfee®

Network Security Manager.

Best methods for rule set creation

There are two best practice methods employed for creating rule sets.

• General-to-specific rule creation — The first method is general-to-specific. Start with an include rule that covers a broad range of operating systems, applications and protocols. After this, create one or more exclude rules to strip away specific operating systems, protocols, et cetera, thus focusing the rule set on the environment where it will be enforced. For example, start with an include rule for all Exploit category attacks. Follow this with multiple exclusion rules that strip away protocols, applications, severities, et cetera, that are rarely or never seen in a zone of your

network.

• Collaborative rule creation — The second method is collaboration: Create multiple include rules within one rule set for each category, operating systems, et cetera, combination that needs to be detected. Each criterion must be matched in order for an alert to be triggered. For example, create the first rule in the set with the Exploit category, Unix as the OS, Sendmail as the application, and SMTP as the protocol. Next, create another include rule for Exploit, Windows 2000, WindMail, and so forth in the same manner. Each include rule added, broadens the scope of the detection. For more information, see Managing Rule Sets, McAfee Network Security Platform IPS

Administration Guide.

(28)

9

How to create rule sets

(29)

10

Working with firewall policies

Review the following points while working with Firewall policies:

• You cannot set explicit access rules for protocols that negotiate ports dynamically, with the

exception of FTP, TFTP, and RPC services. Protocols such as H.323 and Netmeeting, which negotiate the data channel separately from the control channel, or negotiate ports that do not follow a standard, are not supported. However, you can explicitly deny these protocol instances by denying the fixed control port. However, you can configure access rules to explicitly deny these protocol instances by denying the fixed control port.

• For RPC services, you can configure explicit permit and deny rules for RPC as a whole, but not its constituents, such as statd and mountd.

• Protocols or services, such as instant messaging and peer-to-peer communication, that use dynamic ports, are not supported.

• An alternative option for denying protocols that use dynamic ports is to configure IDS policies to drop the attacks that are detected in such transmissions. Network Security Platform detects use of and attacks in such programs as Yahoo Messenger, KaZaA, IRC, and so on.

• There is a limit on the number of access rules that can be supported by various Sensor models. For more information, see McAfee Network Security Platform IPS Administration Guide

(30)

(31)

11

How to handle asymmetric networks

Traffic that uses a different path for the request vs. response is termed as asymmetric traffic. There are chances of having asymmetric traffic within a network, when networks increase in size.

If there are chances of asymmetric traffic in your network, consider the following options: • Install IPS Sensors at a location where the traffic is symmetric.

• Implement a port clustering configuration for asymmetric traffic. Port clustering [referred to as Interface groups in the Manager] enables multiple ports on a single Sensor to be grouped together for effective traffic monitoring. Asymmetric networks are common in load balancing and active/ passive configurations, and a complete transmission may be received on one segment, but depart on another. Thus keeping state of asymmetric transmissions is essential for successfully monitoring the traffic. Interface groups normalize the impact of traffic flows split across multiple interfaces, thus maintaining state to avoid information loss.

• Place an IPS Sensor each on the request and the response path of the asymmetric traffic and create a failover pair to sync up the traffic flow between the two Sensors.

• If you are using a failover pair to monitor asymmetric traffic where the TCP traffic is going through two geographically different data centers, connect the Sensors using dark fiber. In this option, both the Sensors will have full state.

• When the distance between the two IPS Sensors is such that a failover pair cannot be created, consider enabling Stateless Inspection. In Stateless Inspection, the Sensor detects attacks without requiring a valid TCP state. This option should be used only when Sensors are placed in a network where the Sensors do not see all packets of a TCP flow like in an asymmetric network

configuration.

When Stateless Inspection is enabled: - ACLs and syn cookie protection cannot be enabled. - HTTP redirection to the Remediation Portal may or may not work depending on your network deployment scenario for example, in a setup where SYN+ACK packets cannot be sent from the Sensor to the client

The diagram below explains about HTTP traffic flow in an asymmetric network between User A and the University Admin server. The outgoing connection flow from User A is through Switch 1, Switch 2, Network Security Sensor 1, Router 1, Internet Service Provider 1, to the Internet connection. The return path for the packet however, is through Internet Service Provider 2, Router 2 etc. If traffic flows by the Sensor in an asymmetric manner as described above, all packets of a TCP flow are not visible to a single Sensor.

In such a scenario, if Stateless Inspection is enabled, the Sensor will inspect packets without having the valid state for the TCP connection. Consequently, it might generate false positives that is, when a single communication flow is divided across paths, each interface will receive and analyze part of the conversation and therefore be susceptible to false positives and false negatives.

(32)

When you enable Stateless Inspection, there are chances of false positives, and the detection accuracy will be lower compared to when the Sensor sees all traffic. McAfee recommends that you use this feature only when network configuration does not allow the Sensor to be placed in locations where it could see all traffic.

(33)

12

SSL best practices

Note that there is a performance impact when using the SSL decryption feature. If there is a lot of outbound SSL traffic from the client to the internet as well, it consumes SSL flows. Therefore, to enable the Sensor to effectively utilize the SSL decryption feature, it is recommended to bypass these outbound SSL traffic using ACL Ignore rules.

Refer to the following sections for the SSL throughput measurements and test methodologies.

SSL decryption feature is not supported on Virtual Sensors such as NS3200, NS3100, IPS-VM600, IPS-VM100, and IPS-VM100-VSS.

Contents

SSL only traffic — throughput: I-series Sensors SSL only traffic — throughput: M-series Sensors

SSL traffic mixed with HTTP 1.1 traffic: I-series Sensors SSL traffic mixed with HTTP 1.1 traffic: M-series Sensors SSL only traffic - throughput: NS-series Sensors

SSL traffic mixed with HTTP 1.1 traffic: NS-series Sensors

SSL only traffic — throughput: I-series Sensors

• Session resumption for 4 out of 5 TCP connections • 128-bit ARC4

• 5 HTTP 1.1 get page requests per TCP connection with a 5K response each

I-2700 I-3000 I-4000 I-4010

Max. SSL Connections / Sec. 325 600 800 1200

Throughput (Mbps) - 1024 bit key length 85 Mbps 155 Mbps 200 Mbps 310 Mbps Throughput (Mbps) - 2048 bit key length 65 Mbps 115 Mbps 125 Mbps 250 Mbps 5 HTTP 1.1 get page requests per TCP connection with a 10K response each

I-2700 I-3000 I-4000 I-4010

Max. SSL Connections / Sec. 300 400 800 800

Throughput (Mbps) - 1024 bit key length 150 Mbps 200 Mbps 400 Mbps 400 Mbps

12

(34)

SSL only traffic — throughput: M-series Sensors

• Session resumption for 4 out of 5 TCP connections

• 5 HTTP 1.1 get page requests per TCP connection with a 10K response each • 128-bit ARC4 M-8000 M-6050 M-4050 M-3050 M-2950 M-2850/ M-2750 Max. SSL Connections / Sec. 8500 4500 2700 1300 750 550 Throughput (Mbps) - 1024

bit key length 3.8 Gbps 2 Gbps 1200 Mbps 600 Mbps 400 Mbps 250 Mbps Throughput (Mbps) - 2048

bit key length 1.2 Gbps 600 Mbps 550 Mbps 320 Mbps 320 Mbps 200 Mbps

SSL traffic mixed with HTTP 1.1 traffic: I-series Sensors

• 5 HTTP 1.1 get page requests per TCP connection with a 5K response each • 1024-bit RSA

• 128-bit ARC4 I-2700

Max. SSL Connections / Sec. 100 200

SSL Throughput 25 Mbps 50 Mbps

HTTP 1.1 Throughput 475 Mbps 350 Mbps

Total Throughput 500 Mbps 400 Mbps

I-3000

I-4000

I-4010

12

SSL best practices

(35)

I-4010

SSL traffic mixed with HTTP 1.1 traffic: M-series Sensors

• 5 HTTP 1.1 get page requests per TCP connection with a 5K response each • 128-bit ARC4

M-8000

1024 bit key length 2048 bit key length

HTTP 1.1 Throughput 8 Gbps 7.9 Gbps

Total Throughput 8.8 Gbps 8.6 Gbps

M-6050

HTTP 1.1 Throughput 4 Gbps 3.9 Gbps

M-4050

HTTP 1.1 Throughput 2.5 Gbps 2.5 Gbps

M-3050

M-2950

(36)

M-2850/M-2750

SSL Throughput 50 Mbps 40Mbps

SSL only traffic - throughput: NS-series Sensors

NS9300

1024 bit key length 2048 bit key length Max. SSL Connections / Sec. 44000 30800

SSL Throughput 20 Gbps 12 Gbps

NS9200

NS9100

SSL Throughput 8 Gbps 5.5 Gbps

NS7300

NS7200

12

(37)

NS7100

SSL Throughput 1.5 Gbps 1.5 Gbps

NS5200

1024 bit key length 2048 bit key length Max. SSL Connections / Sec. 2,000 2,000

NS5100

SSL traffic mixed with HTTP 1.1 traffic: NS-series Sensors

NS 9300

HTTP 1.1 Throughput 36 Gbps 36 Gbps

Total Throughput 40 Gbps 40 Gbps

NS 9200

NS 9100

(38)

NS 7300

NS 7200

NS 7100

NS5200

NS5100

12

(39)

13

Sensor HTTP response processing

deployment

HTTP response processing is disabled by default. You can enable it for each traffic direction on an interface pair. To minimize the potential performance impact on the McAfee®_{Network Security Sensor}

(Sensor), we recommend that you enable HTTP response processing on the minimum number of ports and in only the required directions to achieve your protection goals.

Some examples of HTTP response processing deployment:

• You want to protect a bunch of clients on your internal network - enable HTTP response processing for inbound traffic only.

• You are serving Web content to external clients, and do not wish to serve attacks embedded in HTTP response traffic - enable HTTP response processing for outbound traffic only.

• You want to protect both internal clients as well as the Web content you are serving to external clients- enable HTTP response processing in both directions.

Tests for enabling HTTP response traffic

The test results provided in the next two sections illustrate potential impact of enabling response processing traffic.

The things to note about the test are given below.

• The test involves only HTTP traffic. Changing the HTTP response processing setting does not change the Sensor performance for any other protocol. Therefore, changes in aggregate Sensor performance will depend on the proportion of HTTP traffic to other traffic on the link being monitored.

• The test sends equal HTTP request and response loads in both directions through the Sensor. Typical real-world deployments do not have equal amounts of HTTP request traffic and response traffic in both directions through the Sensor. Usually, there is significant amount of request traffic in one direction and response traffic in the opposite direction. Since HTTP requests are typically <= 1/10th of the response size, the combined HTTP request and response traffic processed by Sensors in real deployments is typically less than that shown in the tests.

(40)

• The test sends HTTP request continuously at maximum load. Real-world networks are typically loaded, occasionally peaking at maximum capacity, but typically running at significantly lower throughput. The test results reflect performance at sustained load. When not running at maximum load, the Sensor can absorb larger bursts without significant impact.

• The test environment was created to illustrate the likely worst-case performance impact, expected to occur in deployments protecting large Web server farms. In these deployments, HTTP response processing typically provides little value because all HTTP response traffic is sourced from trusted servers, which do not usually transmit hostile content due to the security measures taken. In these environments, customers can consider selectively enabling HTTP response processing to better optimize their network.

The net result of all of these factors is that in typical networks, the impact of enabling HTTP response processing is not noticed. The exact impact is, of course, dependent on the traffic being inspected and some environments could see a reduction in performance as significant as the test results indicate. The factors to take into account include:

• proportion of HTTP traffic to other protocols

• relative amount of HTTP requests and responses in each direction and,

• size of a response page sent to the client by the sites or applications that are typically accessed. For Sensor performance numbers under the following conditions:

• HTTP response processing enabled/disabled and

• 5 HTTP 1.1 get page requests per TCP connection with a 10K response each sent in one direction,

HTTP response processing results for I-series Sensors

Refer to the following table for I-series Sensor performance numbers with HTTP response processing: Model No. HTTP Response Scanning Disabled HTTP Response Scanning Enabled for

outbound direction 5 HTTP 1.1 get page requests per TCP

connection with a 10K response each 5 HTTP 1.1 get page requests per TCPconnection with a 10K response each

I-4010 2 Gbps 1 Gbps I-4000 1.78 Gbps 1 Gbps I-3000 1 Gbps 680 Mbps I-2700 550 Mbps 430 Mbps I-1400 195 Mbps 160 Mbps I-1200 97 Mbps 75 Mbps

HTTP response processing results for M-series Sensors

Refer to the following table for M-series Sensor performance numbers with HTTP response processing: Model No. HTTP Response Scanning Disabled HTTP Response Scanning Enabled for

outbound direction 5 HTTP 1.1 get page requests per TCP

M-8000 10 Gbps 5.4 Gbps

13

Sensor HTTP response processing deployment

(41)

Model No. HTTP Response Scanning Disabled HTTP Response Scanning Enabled for outbound direction M-3050 1.5Gbps 1 Gbps M-2950 1.0 Gbps 850 Mbps M-2850 600 Mbps 500 Mbps M-2750 600 Mbps 500 Mbps M-1450 200 Mbps 200 Mbps M-1250 100 Mbps 100 Mbps

HTTP response processing results for Virtual IPS Sensor

Refer to the following table for Virtual IPS Sensor performance numbers with HTTP response processing:

Model No. HTTP Response Scanning Disabled HTTP Response Scanning Enabled for outbound direction

5 HTTP 1.1 get page requests per TCP

IPS-VM600 600 Mbps 600 Mbps

IPS-VM100 100 Mbps 100 Mbps

HTTP response processing results for NS-series Sensors

Refer to the following table for NS-series Sensor performance numbers with HTTP response processing:

Model No. HTTP Response Scanning Enabled for outbound direction

5 HTTP 1.1 get page requests per TCP connection with a 10K response each NS9300 40 Gbps NS9200 20 Gbps NS9100 10 Gbps NS7300 5 Gbps NS7200 3 Gbps NS7100 1.5 Gbps NS5200 1 Gbps NS5100 600 Mbps NS3200 200 Mbps NS3100 100 Mbps

The NS-series performance numbers when HTTP response is disabled will be higher. For example, the NS9100 performance with HTTP response scanning disabled will be higher than 10 Gbps.

(42)

13

(43)

14

Sensor performance with Layer 7 Data

Collection

Turning on the Layer 7 Data Collection feature reduces Sensor performance. • HTTP Response Scanning setting

• Proportion of HTTP traffic to other protocols

• Relative number of HTTP requests and responses in each direction

• Size of a response page sent to the client by the sites or applications that are typically accessed The following table provides the performance details in a test environment.

• The test environment used 5 HTTP 1.1 get page requests per TCP connection with a 10 K response, each sent in one direction.

• When Advanced Traffic Inspection is enabled, in a deployment with 90 percent of traffic without evasions and 10 percent of traffic with evasions, the overall Sensor throughput would further drop by an additional five percent approximately. For example , if you get 1 Gbps throughput with Layer 7 Data Collection enabled, you would see 950 Mbps if Advanced Traffic Inspection is also enabled.

NS-series Sensor performance with Layer 7 Data Collection

Since the default value of L7 data collection is set to 20% of all traffic, the number of flows decreases by approximately 15%.

Table 14-1 NS9x00 performance details with respect to Layer 7 Data Collection

Sensor Model Layer 7 Data Collection setting HTTP Response

Scanning setting Observed throughput

NS9300 Disabled Disabled 40 Gbps

Enabled for outbound

direction 40 Gbps Percentage of flows that capture

L7 data: 5 Disabled 40 Gbps

direction 40 Gbps

(44)

Table 14-1 NS9x00 performance details with respect to Layer 7 Data Collection (continued)

Scanning setting Observed throughput Enabled for outbound

direction 20 Gbps

direction 10 Gbps

direction 5 Gbps

direction 3 Gbps

direction 1.5 Gbps Percentage of flows that capture

direction 1.5 Gbps

(45)

direction 1 Gbps

NS5100 Disabled Disabled 600 Mbps

direction 600 Mbps Percentage of flows that capture

L7 data: 5 Disabled 600 Mbps

direction 600 Mbps

direction 200 Mbps

direction 100 Mbps

(46)

Virtual IPS Sensor performance with Layer 7 Data Collection

Table 14-5 Sensor performance details with respect to Layer 7 Data Collection

Sensor model Layer 7 Data Collection setting HTTP Response

IPS-VM600 Disabled Disabled 600 Mbps

direction 350 Mbps

IPS-VM100 Disabled Disabled 100 Mbps

direction 85 Mbps

M-series Sensor performance with Layer 7 Data Collection

Table 14-6 Sensor performance details with respect to Layer 7 Data Collection

Sensor model Layer 7 Data Collection setting HTTP Response

Scanning setting Observedthroughput

M-8000 Disabled Disabled 10 Gbps

L7 data: 100 Disabled 8.7 Gbps

direction 4.2 Gbps

M-6050 Disabled Disabled 5 Gbps