LogLogic Cisco NetFlow Log Configuration Guide

(1)

LogLogic Cisco NetFlow



Log Configuration Guide

Document Release: March 2012 Part Number: LL600068-00ELS090000

(2)

This document contains proprietary and confidential information of LogLogic, Inc. and its licensors.  In accordance with the license, this document may not be copied, disclosed, modified, transmitted, or translated except as permitted in writing by LogLogic, Inc.

Trademarks

LogLogic and the LogLogic logo are trademarks or registered trademarks of LogLogic, Inc. in the United States and/or foreign countries. All other company or product names are trademarks or registered trademarks of their respective owners.

Notice

The information contained in this document is subject to change at any time without notice. All

warranties with respect to the software and accompanying documentation are set our exclusively in the Software License Agreement or in the Product Purchase Agreement that covers the documentation.

LogLogic, Inc.

(3)

Cisco NetFlow Log Configuration Guide 3

LogLogic is committed to the success of our customers and to ensuring our products improve customers' ability to maintain secure, reliable networks. Although LogLogic products are easy to use and maintain, occasional assistance might be necessary. LogLogic provides timely and comprehensive customer support and technical assistance from highly knowledgeable,

experienced engineers who can help you maximize the performance of your LogLogic Appliances.

To reach LogLogic Customer Support:

Telephone: Toll Free, US—1 800 957 LOGS (5647) Toll—1 408 834 7480

Telephone: Toll Free, Canada—1 800 957 LOGS (5647) Toll—1 408 834 7480

Telephone: Toll Free, Mexico—1 800 957 LOGS (5647) Toll—1 408 834 7480

Telephone: Toll Free, United Kingdom—00 800 0330 4444 Toll—01480 479391

Telephone: Toll Free, Mainland Europe—00 800 0330 4444 Toll— +44 1480 479391

Telephone: Toll Free, Japan IDC—0061 800 0330 4444 Toll— Not Available

Telephone: Toll Free, Japan KDD—0010 800 0330 4444 Toll— Not Available

Telephone: Toll Free, Brazil—0021 800 0330 4444 Toll— Not Available

Email: [email protected]

You can also visit the LogLogic Support website at: http://www.loglogic.com/services/support.  When contacting Customer Support, be prepared to provide:

Your name, email address, phone number, and fax number Your company name and company address

Your machine type and release version

(6)

Documentation Support

Your feedback on LogLogic documentation is important to us. Send e-mail to

[email protected] if you have questions or comments. Your comments will be reviewed and addressed by the LogLogic technical writing team.

In your e-mail message, please indicate the software name and version you are using, as well as the title and document date of your documentation.

Conventions

LogLogic documentation uses the following conventions to highlight code and command-line elements:

A monospace font is used for programming elements (such as code fragments, objects, methods, parameters, and HTML tags) and system elements (such as filenames, directories, paths, and URLs).

A monospace bold font is used to distinguish system prompts or screen output from user responses, as in this example:

username: system

home directory: home\app

A monospace italic font is used for placeholders, which are general names that you replace with names specific to your site, as in this example: 

LogLogic_home_directory\upgrade\

(7)

Chapter 1 – Configuring LogLogic’s Cisco NetFlow

Log Collection

This chapter describes configuration steps involved to enable a LogLogic Appliance to capture Cisco NetFlow logs. The configuration steps assume that you have a functioning LogLogic Appliance that can be configured to capture Cisco NetFlow log data.

Introduction to Cisco NetFlow . . . 7

Prerequisites . . . 7

Enabling a Cisco Device to Send NetFlow Data . . . 7

Enabling the LogLogic Appliance to Capture Data . . . 8

Verifying the Configuration . . . 10

Introduction to Cisco NetFlow

Cisco NetFlow provides IP application services, plus valuable information about network users and applications, peak usage times, and traffic routing.

Prerequisites

Prior to configuring Cisco NetFlow and the LogLogic Appliance, ensure that you meet the following prerequisites:

Cisco networking device with a NetFlow-enabled IOS. (Cisco 2900, 3500, 3660, and 3750 do not support NetFlow.) See Cisco NetFlow Technical Overview here.

LogLogic Appliance running v5.1 or later with the Cisco NetFlow Log Source Package Administrator access on the LogLogic Appliance

Enabling a Cisco Device to Send NetFlow Data

To configure a Cisco Device to send NetFlow data you will need to use the ip flow-export command through the Cisco’s CLI.

The following example shows the commands to configure the NetFlow version, IP, and port. Router# configure terminal

Router(config)# ip flow-export version 9

Router(config)# ip flow-export destination 10.0.0.1 9995

(8)

Enabling the LogLogic Appliance to Capture Data

The following sections describe how to configure the LogLogic Appliance to capture Cisco NetFlow log data.

Note: When configuring the NetFlow device be sure that you have enabled the proper UDP port in the LogLogic Appliance Access Control list, if Access Control is enabled.

Adding a Cisco NetFlow Device

The LogLogic Appliance captures Cisco NetFlow logs using the NetFlow Collector. You must configure the Cisco NetFlow device with the correct version and port to make the logs available for searching.

To add Cisco NetFlow as a new device:

1. Log in to the LogLogic Appliance.

2. From the navigation menu, select Management > Devices. The Devices tab appears.

3. Click Add New.

The Add Device tab appears.

4. Type in the following information for the device: Name—Name for the Cisco NetFlow device

Description (optional)—Description of the Cisco NetFlow device Device Type—Select Cisco NetFlow from the drop-down menu Host IP—IP address of the Cisco NetFlow appliance

Enable Data Collection—Select the Yes radio button

Refresh Device Name through DNS Lookups (optional)—Select this checkbox to enable the Name field to be automatically updated. The name is obtained using a reverse DNS lookup on the configured refresh interval. The DNS name overrides any manual name you assign.

Cisco NetFlow Collector Configuration— Incoming Port – The port of the Appliance where the NetFlow data for this log source is directed. The port is chosen from a menu that offers port numbers 2055, 9555, and 9995. Although NetFlow devices can usually be configured to any port number, this collector restricts to these three choices so as to work with the LogLogic LMI “Access Control” facility. Note that if “Access Control” is used, any ports used by NetFlow must be configured in the Administration > Firewall Settings configuration page.

Raw Data Forwarding Host (optional) – IP address of the destination host. Raw Data Forwarding Port (optional) – NetFlow port to forward to.

Note: The Raw Data Forwarding feature is used to forward raw NetFlow data to any 3rd party NetFlow receiver in parallel to NetFlow collection on the LogLogic Appliance. This feature is global and applies to all NetFlow data received on the configured Incoming Port.

(9)

5. Click Add.

Figure 1 Adding a Device to the LogLogic Appliance

6. Verify that your new device appears in the Devices tab and that Enabled is set to Yes.

Figure 2 Cisco NetFlow Device Added to LogLogic Appliance Device List

(10)

Verifying the Configuration

The section describes how to verify that the configuration changes made to Cisco NetFlow and the LogLogic Appliance are applied correctly.

To verify the configuration:

1. Log in to the LogLogic Appliance.

2. From the navigation menu, select Dashboards > Log Source Status. The Log Source Status tab appears.

3. Locate the IP address for each Cisco NetFlow device.

If the device name (Cisco NetFlow) appears in the list of devices, then the configuration is correct. If the device does not appear in the Log Source Status tab, run the “show ip flow export” command from the CLI of the Cisco device. Confirm that one of the destinations is the LogLogic Appliance and has the correct Port number and Version.

(11)

Chapter 2 – How LogLogic Supports Cisco NetFlow

This chapter describes LogLogic’s support for Cisco NetFlow. The LogLogic Appliance enables you to capture log data to monitor Cisco NetFlow events.

How LogLogic Captures Cisco NetFlow Log Data . . . 11 LogLogic Real-Time Reports . . . 12

How LogLogic Captures Cisco NetFlow Log Data

A collector is required to listen for the log data from the Cisco NetFlow device as the data is transmitted in binary format. The Cisco NetFlow Collector collects the log data from the Cisco NetFlow device in real time and sends database logs to the LogLogic Appliance.

Figure 4 shows how Cisco NetFlow logs are captured and forwarded to the LogLogic Appliance for further processing.

Figure 4 Cisco NetFlow with LogLogic Components and Processes for Real-Time Collection

(12)

LogLogic Real-Time Reports

LogLogic provides pre-configured Real-Time Reports for Cisco NetFlow log data. The following Real-Time Reports are available:

Application Usage—Displays application usage seen across all traffic User Browsing Statics—Displays site destination statistics by user Top Users—Displays top traffic users

To access LMI 5 Real-Time Reports:

1. In the top navigation pane, click Reports.

2. Click Flow Activity.

The following Real-Time Reports are available: Application Usage

User Browsing Statics Top Users

3. Click Operational.

The following Real-Time Reports are available: All Unparsed Events

(13)

Chapter 3 – Troubleshooting and FAQ

This chapter contains troubleshooting information regarding the configuration and/or use of log collection for Cisco NetFlow. It also contains Frequently Asked Questions (FAQ), providing quick answers to common questions.

Recommended Sampling Rate . . . 13 Troubleshooting . . . 13 Frequently Asked Questions . . . 14

Recommended Sampling Rate

The maximum recommended rate for receiving NetFlow data is 500 flows per second. If you are receiving at a higher rate then this, it is recommended to implement a sampling rate on the Cisco device to limit the amount of flows being sent. Below is a sample configuration.

Router(config)# ip cef

Router(config)# flow-sampler-map my-map

Router(config-sampler)# mode random one-out-of 100 Router(config)# interface GigabitEthernet0/0

Router(config-if)# no ip route-cache flow Router(config-if)# ip route-cache cef Router(config-if)# flow-sampler my-map

This configuration will send 1 out of every 100 NetFlow messages to the LogLogic Appliance. Set the appropriate ratio based on the real-life flow data, but do not exceed 500 flows per second.

Troubleshooting

Problems Retrieving Log Files Using Configured Collector

If you are having general problems retrieving log files using your configured collector, you can run an Index Search against as follows:

1. In the navigation menu, click Search > Index Search.

2. Specify LogLogic Appliance as the Device Type and choose the appropriate Source Device.

(14)

Frequently Asked Questions

How does the LogLogic Appliance obtain the data from the Cisco NetFlow stream?

LogLogic’s Cisco NetFlow Collector runs on the LogLogic Appliance and listens on the specified port for the binary NetFlow stream from a Cisco NetFlow-enabled device.

What access permissions are required?

To configure a Cisco device to send a NetFlow stream, the user must have the proper permissions to make configuration changes to the Cisco device.

How do I know what version and port NetFlow is sending on?

Log into the Cisco device and run the “show ip flow export” command. The following is an example output:

Flow export v5 is enabled for main cache Export source and destination details : VRF ID : Default

Destination(1) 10.1.1.1 (9995) Version 5 flow records

73909 flows exported in 20903 udp datagrams 0 flows failed due to lack of export packet 24 export packets were sent up to process level 0 export packets were dropped due to no fib

0 export packets were dropped due to adjacency issues

0 export packets were dropped due to fragmentation failures

(15)

Appendix A – Event Reference

This appendix lists the LogLogic-supported Cisco NetFlow events. The Cisco NetFlow event table identifies events that can be analyzed through LogLogic reports. All sample log messages were captured by LogLogic’s file pull functionality.

LogLogic Support for Cisco NetFlow Events

The following list describes the contents of each of the columns in the table below. Version – Refers to the log format version

Agile Reports/Search – Defines if the Cisco NetFlow event is available through the LogLogic Agile Report Engine or through the search capabilities. If the event is available through the Agile Report Engine, then you can use LogLogic’s Real-Time Reports and Summary Reports to analyze and display the captured log data. Otherwise, all other

supported events that are captured by the LogLogic Appliance can be viewed by performing a search for the log data.

Title/Comments—Not Applicable (N/A)

Event Category—Event classification (e.g., IN/OUT)

(16)

Table 1 Cisco NetFlow Events Version Agile Reports /Search Event Category Report Appears in

Sample Log Message

1 5 Agile IN Application Usage,

User Browsing Statics, Top Users

<189>[NetFlow] version="5",sysUptime="194642940",unixSecs="2010-03-24T16:37:04",unixNsecs=" 690546564",flowSequence="33398",engineType="0",engineId="0",samplingInterval= "0",IN_BYTES="",IN_PKTS="",FLOWS="",PROTOCOL="1",TCP_FLAGS="16",L4_S RC_PORT="0",IPV4_SRC_ADDR="10.1.70.163",INPUT_SNMP="1",L4_DST_PORT ="771",IPV4_DST_ADDR="10.60.0.140",OUTPUT_SNMP="0",SRC_AS="0",DST_A S="0",MUL_DST_PKTS="",MUL_DST_BYTES="",LAST_SWITCHED="194616940", FIRST_SWITCHED="194616940",OUT_BYTES="",OUT_PKTS="",MIN_PKT_LNGT H="",MAX_PKT_LNGTH="",IPV6_SRC_ADDR="",IPV6_DST_ADDR="",SAMPLING _INTERVAL="",SAMPLING_ALGORITHM="",FLOW_ACTIVE_TIMEOUT="",FLOW_ INACTIVE_TIMEOUT="",TOTAL_BYTES_EXP="224",TOTAL_PKTS_EXP="1",TOT AL_FLOWS_EXP="",SRC_VLAN="",DST_VLAN="",IF_NAME="",IF_DESC="",DST_ MASK="24",IPV4_NEXT_HOP="0.0.0.0",SRC_MASK="8",SRC_TOS="192"

2 9 Agile IN Application Usage, User Browsing Statics, Top Users

<189>[NetFlow] version="9",sysUptime="281117940",unixSecs="2010-03-25T16:38:19",packetSeque nce="192",sourceId="0",IN_BYTES="229",IN_PKTS="1",FLOWS="",PROTOCOL="1 7",TCP_FLAGS="16",L4_SRC_PORT="138",IPV4_SRC_ADDR="10.60.0.31",INPU T_SNMP="1",L4_DST_PORT="138",IPV4_DST_ADDR="10.60.255.255",OUTPUT_ SNMP="0",SRC_AS="",DST_AS="",MUL_DST_PKTS="",MUL_DST_BYTES="",LAS T_SWITCHED="281091296",FIRST_SWITCHED="281091296",OUT_BYTES="",OU T_PKTS="",MIN_PKT_LNGTH="",MAX_PKT_LNGTH="",IPV6_SRC_ADDR="",IPV6 _DST_ADDR="",SAMPLING_INTERVAL="",SAMPLING_ALGORITHM="",FLOW_A CTIVE_TIMEOUT="",FLOW_INACTIVE_TIMEOUT="",TOTAL_BYTES_EXP="",TOT AL_PKTS_EXP="",TOTAL_FLOWS_EXP="",SRC_VLAN="",DST_VLAN="",IF_NAM E="",IF_DESC="",DIRECTION="ingress",DST_MASK="0",FLOW_SAMPLER_ID="0" ,IPV4_NEXT_HOP="0.0.0.0",SRC_MASK="0",SRC_TOS="0",UNKNOWN_51="0"

3 9 Agile OUT Application Usage,

<189>[NetFlow] version="9",sysUptime="281117940",unixSecs="2010-03-25T16:38:19",packetSeque nce="192",sourceId="0",IN_BYTES="229",IN_PKTS="1",FLOWS="",PROTOCOL="1 7",TCP_FLAGS="16",L4_SRC_PORT="138",IPV4_SRC_ADDR="10.60.255.255",IN PUT_SNMP="1",L4_DST_PORT="138",IPV4_DST_ADDR="10.60.0.31",OUTPUT_ SNMP="0",SRC_AS="",DST_AS="",MUL_DST_PKTS="",MUL_DST_BYTES="",LAS T_SWITCHED="281091296",FIRST_SWITCHED="281091296",OUT_BYTES="",OU T_PKTS="",MIN_PKT_LNGTH="",MAX_PKT_LNGTH="",IPV6_SRC_ADDR="",IPV6 _DST_ADDR="",SAMPLING_INTERVAL="",SAMPLING_ALGORITHM="",FLOW_A CTIVE_TIMEOUT="",FLOW_INACTIVE_TIMEOUT="",TOTAL_BYTES_EXP="",TOT AL_PKTS_EXP="",TOTAL_FLOWS_EXP="",SRC_VLAN="",DST_VLAN="",IF_NAM E="",IF_DESC="",DIRECTION="ingress",DST_MASK="0",FLOW_SAMPLER_ID="0" ,IPV4_NEXT_HOP="0.0.0.0",SRC_MASK="0",SRC_TOS="0",UNKNOWN_51="0"

4 5 Agile OUT Application Usage,

(17)

Appendix B – Field Descriptions

This appendix lists the field descriptions for the LogLogic-supported Cisco NetFlow events, examples of which appear in Appendix A above.

Table 2 Filed Descriptions for Cisco NetFlow v5.0

Netflow v5 Fields Description

version The version of NetFlow records exported in this packet; for Version 9, this value is 0x0009 sysUptime SysUptime Time in milliseconds since this device was first booted

unixSecs UnixSecs Seconds since 0000 Coordinated Universal Time (UTC) 1970 unixNsecs Residual nanoseconds since 0000 UTC 1970

flowSequence Sequence counter of total flows seen engineType Type of flow-switching engine

engineId Slot number of the flow-switching engine

samplingInterval First two bits hold the sampling mode; remaining 14 bits hold value of sampling interval IN_BYTES Incoming counter with length N x 8 bits for number of bytes associated with an IP Flow IN_PKTS Incoming counter with length N x 8 bits for the number of packets associated with an IP Flow

FLOWS Number of flows that were aggregated

PROTOCOL IP protocol byte

TCP_FLAGS Cumulative of all the TCP flags seen for this flow

L4_SRC_PORT TCP/UDP source port number ie : FTP, Telnet, or equivalent IPV4_SRC_ADDR IPv4 source address

INPUT_SNMP Input interface index;

L4_DST_PORT TCP/UDP destination port number ie: FTP, Telnet, or equivalent IPV4_DST_ADDR IPv4 destination address

OUTPUT_SNMP Output interface index;

SRC_AS Source BGP autonomous system number

DST_AS Destination BGP autonomous system number

MUL_DST_PKTS IP multicast outgoing packet counter with length N x 8 bits for packets associated with the IP Flow

MUL_DST_BYTES IP multicast outgoing byte counter with length N x 8 bits for bytes associated with the IP Flow

LAST_SWITCHED System uptime at which the last packet of this flow was switched FIRST_SWITCHED System uptime at which the first packet of this flow was switched

OUT_BYTES Outgoing counter with length N x 8 bits for the number of bytes associated with an IP Flow OUT_PKTS Outgoing counter with length N x 8 bits for the number of packets associated with an IP Flow MIN_PKT_LNGTH Minimum IP packet length on incoming packets of the flow

MAX_PKT_LNGTH Maximum IP packet length on incoming packets of the flow IPV6_SRC_ADDR IPv6 Source Address

IPV6_DST_ADDR IPv6 Destination Address

SAMPLING_INTERVAL When using sampled NetFlow, the rate at which packets are sampled ie: a value of 100 indicates that one of every 100 packets is sampled

(18)

FLOW_ACTIVE_TIMEOUT Timeout value (in seconds) for active flow entries in the NetFlow cache FLOW_INACTIVE_TIMEOUT Timeout value (in seconds) for inactive flow entries in the NetFlow cache

TOTAL_BYTES_EXP Counter with length N x 8 bits for bytes for the number of bytes exported by the Observation Domain

TOTAL_PKTS_EXP Counter with length N x 8 bits for packets for the number of bytes exported by the Observation Domain

TOTAL_FLOWS_EXP Counter with length N x 8 bits for flows for the number of bytes exported by the Observation Domain

SRC_VLAN Virtual LAN identifier associated with ingress interface DST_VLAN Virtual LAN identifier associated with egress interface

IF_NAME Name of the interface

IF_DESC Full interface name ie: "'FastEthernet 1/0" DST_MASK Destination address prefix mask bits

IPV4_NEXT_HOP Next Hop

SRC_MASK Source address prefix mask bits SRC_TOS Source IP type of service (ToS)

version The version of NetFlow records exported in this packet; for Version 9, this value is 0x0009 sysUptime SysUptime Time in milliseconds since this device was first booted

unixSecs UnixSecs Seconds since 0000 Coordinated Universal Time (UTC) 1970

packetSequence Incremental sequence counter of all export packets sent by this export device; this value is cumulative, and it can be used to identify whether any export packets have been missed sourceId The Source ID field is a 32-bit value that is used to guarantee uniqueness for all flows

exported from a particular device.

IN_BYTES Incoming counter with length N x 8 bits for number of bytes associated with an IP Flow IN_PKTS Incoming counter with length N x 8 bits for the number of packets associated with an IP Flow

FLOWS Number of flows that were aggregated

PROTOCOL IP protocol byte

TCP_FLAGS Cumulative of all the TCP flags seen for this flow

L4_SRC_PORT TCP/UDP source port number ie : FTP, Telnet, or equivalent IPV4_SRC_ADDR IPv4 source address

INPUT_SNMP Input interface index;

L4_DST_PORT TCP/UDP destination port number ie: FTP, Telnet, or equivalent IPV4_DST_ADDR IPv4 destination address

OUTPUT_SNMP Output interface index;

SRC_AS Source BGP autonomous system number

DST_AS Destination BGP autonomous system number

MUL_DST_PKTS IP multicast outgoing packet counter with length N x 8 bits for packets associated with the IP Flow

MUL_DST_BYTES IP multicast outgoing byte counter with length N x 8 bits for bytes associated with the IP Flow

(19)

LAST_SWITCHED System uptime at which the last packet of this flow was switched FIRST_SWITCHED System uptime at which the first packet of this flow was switched

OUT_BYTES Outgoing counter with length N x 8 bits for the number of bytes associated with an IP Flow OUT_PKTS Outgoing counter with length N x 8 bits for the number of packets associated with an IP Flow MIN_PKT_LNGTH Minimum IP packet length on incoming packets of the flow

MAX_PKT_LNGTH Maximum IP packet length on incoming packets of the flow IPV6_SRC_ADDR IPv6 Source Address

IPV6_DST_ADDR IPv6 Destination Address

SAMPLING_INTERVAL When using sampled NetFlow, the rate at which packets are sampled ie: a value of 100 indicates that one of every 100 packets is sampled

SAMPLING_ALGORITHM The type of algorithm used for sampled NetFlow: 0x01 Deterministic Sampling ,0x02 Random Sampling

FLOW_ACTIVE_TIMEOUT Timeout value (in seconds) for active flow entries in the NetFlow cache FLOW_INACTIVE_TIMEOUT Timeout value (in seconds) for inactive flow entries in the NetFlow cache

TOTAL_BYTES_EXP Counter with length N x 8 bits for bytes for the number of bytes exported by the Observation Domain

TOTAL_PKTS_EXP Counter with length N x 8 bits for packets for the number of bytes exported by the Observation Domain

TOTAL_FLOWS_EXP Counter with length N x 8 bits for flows for the number of bytes exported by the Observation Domain

SRC_VLAN Virtual LAN identifier associated with ingress interface DST_VLAN Virtual LAN identifier associated with egress interface

IF_NAME Name of the Interface

IF_DESC Full interface name ie: "'FastEthernet 1/0" DIRECTION Flow direction: 0 - ingress flow, 1 - egress flow DST_MASK Destination address prefix mask bits

FLOW_SAMPLER_ID The Sampling Algo Flow ID

IPV4_NEXT_HOP Next Hop

SRC_MASK Source address prefix mask bits SRC_TOS Source IP type of service (ToS)

(20)

LogLogic Cisco NetFlow Log Configuration Guide