Smart Connector Users Guide

(1)

SmartConnector™

User’s Guide

Topics Applicable to All ArcSight™ SmartConnectors

(2)

ArcSight, the ArcSight logo, ArcSight TRM, ArcSight NCM, ArcSight Enterprise Security Alliance, ArcSight Enterprise Security Alliance logo, ArcSight Interactive Discovery, ArcSight Pattern Discovery, ArcSight Logger, FlexConnector, SmartConnector, SmartStorage and CounterACT are trademarks of ArcSight, Inc. All other brands, products and company names used herein may be trademarks of their respective owners.

Follow this link to see a complete statement of ArcSight's copyrights, trademarks, and acknowledgements: http://www.arcsight.com/company/copyright/

The network information used in the examples in this document (including IP addresses and hostnames) is for illustration purposes only.

This document is ArcSight Confidential. Revision History

Document template version: 2.1

ArcSight Customer Support

Date Description

05/15/2011 Added new CEF encrypted Syslog destination and expanded CEF chapter. 02/15/2011 Added features to CEF Syslog destination and updated installation screens and

procedures, added information on deleting file connectors and improved readability of FAQ appendix.

11/30/2010 Added feedback from reorganization review, corrected and closed various open tickets.

02/24/2010 Added chapter 10, defining the new CEF destination.

09/30/2009 Complete restructure of the guide and the addition of the “Configuring SmartConnectors” chapter.

08/24/2009 Added Model Connector information, new FAQ entries, and updated configuration fields.

Phone 1-866-535-3285 (North America)

+44 (0)870 141 7487 (EMEA)

E-mail support@arcsight.com

Support Web Site http://www.arcsight.com/supportportal/

(3)

About This Book

This book contains information that applies to all SmartConnectors, including installation, deployment, and management of SmartConnectors. Information about installing and configuring individual SmartConnectors is provided in the ArcSight SmartConnector Configuration Guides.

The following topics are discussed in this chapter:

Who Should Read This Book

The audience for this book is primarily security administrators who install SmartConnectors and ensure their connectivity to ArcSight products. This can include administrators for:

 Networks  Security

 Systems

 Databases

If this is the first time you are installing an ArcSight component, ArcSight recommends reading the latest Administrator’s Guide for that component.

“Who Should Read This Book” on page 7 “Related Documentation” on page 8 “ArcSight Customer Support” on page 9

(8)

ArcSight Customer Support

You can obtain a log-in user name and password from your ArcSight Customer Support representative. You can reach ArcSight Customer Support through the following resources:

Resource Description

Support Website http://www.arcsight.com/supportportal/ provides access to ArcSight incident reporting, knowledge base, software downloads, help, and the new Customer Forum. Protect 724

(10)

(11)

Chapter 1 Introduction to ArcSight Products

ArcSight products comprise several separately installable components working together to process event data from your network. These components are connected throughout your network by way of sensors that report to a series of ArcSight SmartConnectors.

SmartConnectors translate device output into a normalized event schema that becomes the starting point for ArcSight ESM correlation. The following graphic illustrates ArcSight basic components. For complete descriptions of these components, see ESM 101, Concepts for ArcSight ESM v4.0.

“ArcSight ESM” on page 12

“ArcSight SmartConnectors” on page 13 “ArcSight FlexConnectors” on page 14 “Arcsight Connector Appliance” on page 15 “ArcSight Logger” on page 15

(12)

 Users interact with ArcSight ESM using the ESM Console or ArcSight Web.

 ArcSight SmartConnectors gather and process event data from network devices and

pass it to the ESM Manager to be processed and stored in the database.

 ArcSight Connector Appliance is a hardware solution incorporating any number of

onboard SmartConnectors and a web-based user interface. This tool provides centralized management for SmartConnectors across a number of hosts.

 ArcSight NSP uses NCM/TRM software to provide network device inventory,

configuration settings, and additional analysis features.

 ArcSight Logger is a hardware storage solution optimized for extremely high event throughput.

Arcsight Components

ArcSight ESM

ArcSight ESM consists of several separately installable components that work together to process event data from your network. These components are described in the following pages.

ESM Manager

As events stream into the system, the ESM Manager writes them to the ArcSight database. It simultaneously processes the events through the correlation engine, which evaluates each event with network model and vulnerability information to develop real time threat summaries.

ESM Database

As events stream into the ESM Manager from the SmartConnectors, they are written to the ESM Database with a normalized schema. This lets ESM collect all events generated by the devices on your network, which you can analyze and refer to at any time.

The ESM Database is based upon Oracle 10g. A typical installation retains active data online from weeks to months.

ESM Console

The ArcSight ESM Console is a workstation-based interface intended for use by your full-time security staff in a Security Operations Center (SOC) or similar security-monitoring environment. The Console is the authoring tool for building ArcSight ESM filters, rules, reports, Pattern Discovery, dashboards, and data monitors. It also is the interface for administering users and resources.

ArcSight Web

ArcSight Web is an independent and remotely installable Web server that provides a secure interface with the ArcSight ESM Manager for browser clients. ArcSight Web is intended for use as a streamlined interface for customers of Managed Service Security Providers (MSSPs), SOC operators, and business users who require access to ArcSight ESM to investigate events from outside the protected network.

The ArcSight ESM Console version should match the ArcSight ESM Manager version to ensure that resources and schemas match.

(13)

1 Introduction to ArcSight Products

ArcSight SmartConnectors

SmartConnectors are the interface between the ArcSight ESM Manager and the network devices that generate ESM-relevant data on your network.

SmartConnectors collect event data from network devices, then normalize it in two ways. First, they normalize values (such as severity, priority, and time zone) into a common format. Then they normalize the data structure into a common schema. SmartConnectors can filter and aggregate the events to reduce the volume sent to the ESM Manager, which increases ArcSight’s efficiency and reduces event processing time.

In brief, SmartConnectors:

 Collect all the data you need from a source device, eliminating the need to return to

the device during an investigation or audit.

 Parse individual events and normalize event values (such as severity, priority and time zone) into a common schema (format) for use by ArcSight ESM.

 Filter out data you know is not needed for analysis, thus saving network bandwidth

and storage space.

 Aggregate events to reduce the quantity of events sent to the Manager, increasing Arcsight’s efficiency and reducing event processing time.

 Pass processed events to the Manager.

 Categorize events using a common, human-readable format, saving you time and

making it easier to use those event categories to build filters, rules, reports, and data monitors.

Depending upon the network device, some SmartConnectors can also instruct the device to issue commands to devices. These actions can be executed manually or through

automated actions from rules and some data monitors.

ArcSight releases new and updated SmartConnectors approximately twice a quarter.

Supported Data Sources

ArcSight collects output from data sources with network devices, such as intrusion detection and prevention systems, vulnerability assessment tools, firewalls, anti-virus and anti-spam tools, encryption tools, application audit logs, and physical security logs. SmartConnectors can be installed either directly on devices or separately on dedicated servers, depending upon the network device reporting to them. The SmartConnector can be co-hosted on the device if the device is a standard PC and its function is entirely software-based, such as IBM/ISS RealSecure devices, Snort devices, and so on. For embedded data sources (such as most Cisco devices and Check Point Firewall appliances), co-hosting on the device is not an option.

During SmartConnector configuration, a SmartConnector is registered to your ArcSight ESM Manager and configured with characteristics unique to the devices it reports on and the business needs of your network.

By default, SmartConnectors maintain a heartbeat with the ESM Manager every 10 seconds. The Manager sends any Console commands or configuration updates to the

(14)

Event Severity

During the normalization process, the SmartConnector collects data about the level of danger associated with a particular event, as interpreted by the data source that reported the event to the SmartConnector. These data points, device severity and

SmartConnector severity, become factors in calculating the event’s overall priority. Device severity captures the language used by the data source to describe its interpretation of the danger posed by a particular event. For example, if a network IDS detects a DHCP packet that does not contain enough data to conform to the DHCP format, the device flags this as a high-priority exploit.

SmartConnector severity is the translation of the device severity into

ArcSight-normalized values. For example, Snort uses a device severity scale of 1-10, whereas Check Point uses a scale of high, medium, and low. ArcSight normalizes these values into a single severity scale. The default ArcSight scale is Low, Medium, High, and Very High.

For example, routine file access and successful authentications by authorized users would be translated into the ArcSight-normalized values as very low severity, whereas a short DHCP packet would be translated as very high severity.

ArcSight FlexConnectors

ArcSight’s FlexConnector framework is a software development kit (SDK) that lets you create a SmartConnector tailored to the devices on your network and their specific event data. The following ArcSight FlexConnectors types are available:

 CounterACT  File  ID-Based Database  Key-Value File  Multiple Database  Multi-Folder File  Regular Expression File  Regular Expression Folder File  Regular Expression Multiple File  SNMP

 Syslog

 Time-Based Database  XML File

In addition, beta support is currently available for the following FlexConnectors:

 Scanner Database

 Scanner XML Reports  Scanner Text Reports

For complete information about these FlexConnectors and how to use them, contact your ArcSight Customer Support representative or see the ArcSight FlexConnector Developer's Guide.

(15)

1 Introduction to ArcSight Products

Arcsight Connector Appliance

ArcSight Connector Appliance is a hardware solution that incorporates a number of onboard ArcSight SmartConnectors and a web-based user interface that provides centralized management for SmartConnectors across a potentially large number of hosts. The Connector Appliance centralizes SmartConnector management and offers unified control of SmartConnectors on local and remote Connector Appliances as well as software-based SmartConnectors installed on remote hosts.

ArcSight Connector Appliance includes on-board SmartConnectors that connect event sources to destinations such as ArcSight Logger and ArcSight ESM.

The Connector Appliance delivers the following features and benefits:

 Supports bulk operations across all SmartConnectors and is particularly desirable in ArcSight ESM deployments with a large number of SmartConnectors, such as a Managed Security Services Provider (MSSP).

 Provides an ArcSight ESM-like SmartConnector management facility in Logger-only environments.

 Provides a single interface through which to configure, monitor, tune, and update

SmartConnectors. The Connector Appliance does not receive events from the SmartConnectors it manages, and this allows for management of many connectors at one time. The Connector Appliance does not affect working SmartConnectors unless it is used to change their configuration. In some cases, the SmartConnector is

commanded to restart.

ArcSight Logger

ArcSight Logger is an event data storage appliance optimized for extremely high event throughput. Logger stores security events onboard in compressed form, but can always retrieve unmodified events on demand for forensics-quality litigation data.

Logger can be deployed stand-alone to receive events from syslog messages or log files, or to receive events in Common Event Format from SmartConnectors. Logger can forward selected events as syslog messages to ESM.

Multiple Loggers work together to support high sustained input rates. Event queries are distributed across a peer network of Loggers.

ArcSight Network Synergy Platforms (NSP)

ArcSight NSP is an appliance that consists of these two licensed software components.

 Network Configuration Manager (NCM)  Threat Response Manager (TRM)

These components build and maintain a detailed understanding of your network’s topology, letting you centrally manage your network infrastructure and rapidly respond to security incidents.

(16)

The NCM/TRM solution lets you:

 Locate and quarantine any device connected to the network instantly  Apply protocol filters to curb an intrusion attempt

 Block specific IP ranges from communicating or block specific protocols  Disable individual user accounts

 Manage configuration changes centrally on a single device or a group of devices  Audit the change control process granularity

 Build wizards that let you to delegate routine network administration tasks to

(17)

Chapter 2 SmartConnector Overview

This chapter provides an overview of ArcSight SmartConnectors and how they collect and send events (generated by various vendor devices) to the ArcSight ESM Manager. The following topics are included in this chapter:

Once SmartConnectors normalize and send events to the ArcSight Manager, the events are stored in the centralized ESM Database. ArcSight ESM then filters and cross-correlates these events with rules to generate meta-events. The meta-events then are automatically sent to administrators with corresponding Knowledge Base articles that contain information supporting their enterprise’s policies and procedures.

SmartConnectors process raw data generated by various vendor devices throughout an enterprise. Devices consist of routers, e-mail servers, anti-virus products, firewalls, intrusion detection systems (IDS), access control servers, VPN systems, anti-DoS appliances, operating system logs, and other sources that detect and report security or audit information.

ArcSight SmartConnectors collect a vast amount of varying, heterogeneous information. Due to this variety of information, SmartConnectors format each event into a consistent, normalized ArcSight message, letting you find, sort, compare, and analyze all events using the same event fields.

Specific SmartConnector Configuration Guides document device-to-ArcSight ESM event mapping information for individual vendor devices, as well as specific installation parameters and configuration information.

“Features” on page 18

“Data Collection Methods” on page 20 “Mapping to Vendor Events” on page 20 “Filter and Aggregate Events” on page 20 “SmartConnector Types” on page 21

(18)

Features

For complete information about how the following features work, see the ArcSight ESM v4.0 Administrator’s Guide and ArcSight ESM Console Help.

The following illustration shows the communication between network devices and ArcSight SmartConnectors, and between ArcSight SmartConnectors and ArcSight ESM Manager.

Feature Description

Filtering and Data

Reduction Uses AND/OR based Boolean logic to determine what data is to be included from the device and what data is filtered out when the event is sent to the ESM Manager.

Aggregation Compiles events with matching values into a single event, reducing the number of individual events the ESM Manager must evaluate.

Batching Improves ESM Manager performance by sending a collection of events at one time (rather than after each occurrence).

Time Error

Correction Synchronizes the time between the device and the SmartConnector, and between the SmartConnector and the ESM Manager.

Time Zone Correction Corrects the local time zone, as necessary, to support device-time queries, correlation, and filters.

Categorizer Assigns ArcSight ESM categories to an event.

Resolver Attempts to resolve and reverse-resolve host names and addresses reported by a device.

Data Normalization Converts each event produced by devices to an ArcSight ESM common event format message (or ArcSight message).

(19)

2 SmartConnector Overview

Figure 2-1 ArcSight SmartConnector Event Collection and Processing

SmartConnectors both receive and retrieve information from network devices. If the device sends information, the SmartConnector becomes a receiver; if the device does not send information, the SmartConnector retrieves it.

An ArcSight message is created for each event the SmartConnectors collect. Once an event is received, the SmartConnector adds device and event information to the event to complete the message, which is then sent to the ESM Manager.

You can deploy SmartConnectors on a device, on a separate host machine, or on the host machine where the ArcSight ESM Manager system resides.

(20)

Data Collection Methods

ArcSight SmartConnectors are specifically developed to interoperate with network and security products using multiple techniques, including simple log forwarding and parsing, direct installation on native devices, SNMP, and syslog.

Data collection and event reporting formats for various SmartConnectors include:

 Log File Readers (including text and log file)  Syslog

 SNMP  Database  XML

 Proprietary protocols, such as OPSEC or Cisco PostOffice

The ArcSight Console, Manager, and SmartConnectors communicate using HTTP

(HyperText Transfer Protocol) over SSL (Secure Sockets Layer; also referred to as HTTPS). Vendor device types for which SmartConnectors are available include:

 Network and host-based IDS and IPS  VPN, Firewall, router, and switch devices

 Vulnerability management and reporting systems  Access and identity management

 Operating systems, Web servers, content delivery, log consolidators, and aggregators

For more information about the latest ArcSight SmartConnectors available, visit our website at http://www.arcsight.com and click the Support link.

Mapping to Vendor Events

ArcSight SmartConnectors collect the vendor-specific event definitions contained within a network device. This information is mapped to the data fields within the SmartConnector, then sent to the ArcSight ESM Manager.

For specific mappings between the SmartConnector data fields and supported vendor-specific event definitions, see the configuration guide for the device-specific SmartConnector. For example, for mappings for the SmartConnector for Cisco PIX Syslog, refer to the SmartConnector for Cisco PIX Syslog Configuration Guide.

For additional information about mappings and parsing information from third-party devices, see “Advanced Topics” in the FlexConnector Developer’s Guide.

Filter and Aggregate Events

During SmartConnector installation and configuration, you can configure the

SmartConnector to use filter conditions to focus the events passed to the ESM Manager according to specific criteria. For example, you can use filters to sort out events with certain characteristics, from specific network devices, or generated by vulnerability scanners. Events that do not meet the SmartConnector filtering criteria are not forwarded to the ESM Manager.

(21)

You can configure the SmartConnector to aggregate (summarize and merge) events that have the same values in a specified set of fields, either for a specified number of times or within a specified time limit.

SmartConnector aggregation compiles events with matching values into a single event. The aggregated event contains only the values the events have in common plus the earliest start time and latest end time. This reduces the number of individual events the Manager must evaluate.

For example, suppose the SmartConnector is configured to aggregate events with a certain Source IP and Port, Destination IP and Port, and Device Action whenever the events occur 10 times in 30 seconds. If ten events with these matching values are received by the SmartConnector within that timeframe, they are grouped together into a single event with an aggregated event count of 10.

If the 30-second timeframe expires and the SmartConnector has received only two matching events, the SmartConnector creates a single aggregated event with an aggregated event count of two. If 900 matching events were to come in during the 30 seconds, the SmartConnector would create 90 aggregated events, each with an aggregated event count of 10.

Firewalls are a good candidate for aggregation because of the volume of events with similar data coming in from multiple devices.

SmartConnector Types

SmartConnectors are the interface between the ArcSight ESM Manager and the network devices that generate ESM-relevant data on your network.

ArcSight SmartConnectors are generally one of the following types:

 File Connectors  Database Connectors  Scanner Connectors  API Connectors  SNMP Connectors

 Microsoft Windows Event Log Connectors  Syslog Connectors

 FlexConnectors  Model Connectors

SmartConnectors collect event data from network devices, then normalize this data in two ways. First, they normalize values (such as severity, priority, and time zone) into a common format. They then normalize the data structure into a common schema. SmartConnectors can filter and aggregate events to reduce the volume sent to the

ESM Manager, which increases ArcSight's efficiency and reduces event processing time. For general information about ArcSight SmartConnectors, see Chapter 1‚ Introduction to

(22)

File Connectors

There are two primary types of log file connector, Real Time and Folder Follower: Real Time

These connectors can continue to follow a log file that retains its name or changes its name based upon the current date and other factors. The type of real time file connector is based upon the number of files monitored by the connector. There are connectors that monitor a single log file, such as the Snort File connector, and connectors that monitor multiple log files, such as the Cisco Secure ACS and SAP Real Time Audit connectors.

Real Time log file connectors can read normal log files in which lines are separated by a new line character as well as fixed length records in which a file consists of only one line but multiple records of fixed length (such as the SAP Real Time Audit connector). Folder Follower

Folder follower connectors can follow files deposited into a single folder. There are connectors that monitor a single log file (such as HP-UX or IBM AIX) and connectors that monitor log files recursively (such as F-Secure AntiVirus).

.txt and .xml file types are supported by ArcSight SmartConnectors; which type depends upon the particular device. Text log files are the most common; however, Tripwire and most of the scanner file connectors, such as Nessus, nCircle, and NeXpose are in xml format.

The type of log file connector is not usually part of the connector name unless both types of connector exist for a particular device (such as SAP Audit and SAP Real-Time Audit). File connectors are normally installed on the device machine, but when the monitored files are accessible through network shares or NFS mounts, the connectors can be installed on remote machines.

For some connectors, a trigger file is required to tell the connector when the file is complete and ready for processing. Typically, this is the same file name with a different extension. Files are renamed by default to increments such as .processed,

.processed.1, and so on.

Generally, the only parameter required at installation is the location of the log file or files (the absolute path). When default file paths are known, they are displayed in the installation wizard.

Deleting Log Files After Processing

If you choose to delete your log files after the SmartConnector has processed them, you can access the connector's advanced parameters as follows:

1 From the $ARCSIGHT_HOME\current\bin directory in a DOS command window, enter: arcsight connectorsetup

To rename or delete log files, file folders require permissions for the connector.

(23)

2 When the following message is displayed, click No.

3 The Agent Configuration Tool window is displayed. From the Options menu, select Show Internal Parameters. The advanced configuration parameters for the connector are displayed as shown in the following figure.

4 To delete log files after processing, change the value for the mode parameter from RenameFileTheSameDirectory to DeleteFile.

5 Click OK.

6 Restart the connector for your change to take effect.

Database Connectors

Database connectors use SQL queries to periodically poll for events. ArcSight

SmartConnectors support major database types, including MS SQL, MS Access, MySQL, Oracle, DB2, Postgres, and Sybase.

In addition to the native JDBC driver for each database type, database connectors allow the use of a JDBC ODBC driver for databases that support them, such as MS SQL, Postgres, and MS Access. To use a JDBC ODBC driver, a JDBC ODBC data source is required. For instructions about creating this data source, see the configuration guide for your database connector.

(24)

During installation, the installation wizard asks, at a minimum, the following parameter values:

 JDBC ODBC Driver  JDBC ODBC Data Source  Database User

 Database Password

The database user must have adequate permission to access and read the database. For Audit database connectors, such as SQL Server Audit DB and Oracle Audit DB, system administrator permission is required.

In addition to connectors supporting event collection from a single database, some database connectors support multiple database events such as the Microsoft SQL Server Multiple DB connector. Others collect events from scanner databases, such as

SmartConnectors for McAfee FoundScan DB and Mazu Profiler. There are three major types of database connector:

Time-Based

Queries use a time field to retrieve events found since the most recent query time until the current time.

ID-Based

Queries use a numerically increasing ID field to retrieve events from the last checked ID until the maximum ID.

Job ID-Based

Queries use Job IDs that are not required to increase numerically. Processed Job IDs are filed in such a way that only new Job IDs are added. Unlike the other two types of database connector, Job IDs can run in either Interactive mode or Automatic mode.

Scanner Connectors

There are two types of scanner connector, those whose results are retained within a file, and those retrieved from a database. Results for the following types of scanner connectors are retained in a file, making them log file connectors:

 XML files (such as Tenable Nessus, nCircle Audit, Qualys Scanner, and Rapid7

NeXpose)

 Text files (such as Tenable Nessus NSR, NetRecon NRD)

Other scanners deposit their scanned events in a database and are treated as database connectors, requiring the same installation parameters as database connectors.

Scan reports are converted into base events, which can be viewed on the ESM Console, and aggregated meta events, which are not shown on the Console. Meta events create assets, asset categories, open ports, and vulnerabilities on the ESM Console.

(25)

Scanner SmartConnectors can run in either of two modes, automatic or interactive. Interactive mode

Displays scan reports that can be individually selected to be sent to the connector. This mode is not supported for a connector running as a service.

Automatic mode

The connector checks periodically for any new reports deposited into the folder or any new jobs inserted into the database, then processes them. This mode is supported for both stand-alone applications and services.

Other than the operating mode, other parameter values required for scanner installation depends upon whether a file or database connector has been implemented. For file connectors, the absolute path to and name of the log file is required. For database connectors, see “Database Connectors” on page 23.

API Connectors

API connectors use a standard or proprietary API to pull events from devices. In most cases, a certificate must be imported from the device to authenticate connector access to the device. There are also a number of configuration steps required on the device side. For example, Check Point devices require the configuration of connection type and the importing of a client certificate.

During installation, the following types of parameters are required, although each device's parameters are specific to its API:

 Device IP  Service Port

 Event types to be pulled  Certificate information

 Information specific to the particular API

SNMP Connectors

SNMP Traps contain variable bindings, each of which holds a different piece of information for the event. They are usually sent over UDP to port 162, although the port can be changed.

SNMP connectors listen on port 162 (or any other configured port) and process the received traps. They can process traps only from one device with a unique Enterprise OID, but can receive multiple trap types from this device.

As with syslog connectors (because SNMP is based upon UDP), there is a slight chance of events being lost over the network.

Parsers use the knowledge of the MIB to map the event fields, but, unlike some other SNMP-based applications, the connector itself does not require the MIB to be loaded. No parameters are required during connector installation for SNMP devices.

(26)

Microsoft Windows Event Log Connectors

System administrators use the Windows Event Log for troubleshooting errors. Each entry in the event log can have a severity of Error, Warning, Information, plus Success Audit or Failure Audit.

There are three default Windows Event Logs:

 Application log (tracks events that occur in a registered application)  Security log (tracks security changes and possible breaches in security)  System log (tracks system events)

There are three SmartConnectors for Microsoft Windows Event Log:

 SmartConnector for Microsoft Windows Event Log – Unified, this connector

can connect to local or remote machines, inside a single domain or from multiple domains, to retrieve and process security and system events.

 SmartConnector for Microsoft Windows Event Log – Local, which collects

events from the Windows Event Log on your local machine.

 SmartConnector for Microsoft Windows Event Log – Domain, which lets you collect Microsoft Windows Event Log events from multiple remote machines and forward them into the ArcSight system (such as multiple occurrences of the same application installed on different machines in one domain).

For details about the Local and Domain connectors deployment, installation, and configuration, see the SmartConnector Configuration Guide for Microsoft Windows Event Log. For mappings, see ArcSight SmartConnector Mappings to Windows Security Events. For details about the Unified connector, see the SmartConnector Configuration Guide for Microsoft Windows Event Log – Unified. Mappings for this connector are incorporated into its configuration guide.

The SmartConnector for Microsoft Windows Event Log – Unified supports event collection from Microsoft Windows XP, Server 2000/2003/2008 and Vista platforms, as well as support for partial event parsing based upon the Windows event header for all System and Application events. Support for a FlexConnector-like framework that lets users create and deploy their own parsers for parsing the event description for all System and Application events is also provided.

Some individual Windows Event Log applications are supported by the SmartConnector for Microsoft Windows Event Log – Domain, for which Windows Event Log sub-connectors have been developed. These sub-connectors have individual configuration guides that provide setup information and mappings for the particular application. These

sub-connectors include:

 CA eTrust AntiVirus Windows Event Log

 Microsoft Active Directory Service Windows Event Log  Microsoft WINS Windows Event Log

 Oracle Audit Windows Event Log  RSA ACE Server Windows Event Log  Symantec Mail Security Windows Event Log

(27)

Syslog Connectors

Syslog messages are free-form log messages prefixed with a syslog header consisting of a numerical code (facility + severity), timestamp, and host name. They can be installed as a syslog daemon, pipe, or file connector. Unlike file connectors, a syslog connector can receive and process events from multiple devices. There is a unique regular expression that identifies the device.

 Syslog Daemon connectors listen for syslog messages on a configurable port, using

port 514 as a default. It is the only syslog option supported for Windows platforms.

 Syslog Pipe connectors require syslog configuration to send messages with a certain syslog facility and severity.

The Solaris platform tends to under perform when using Syslog Pipe connectors. The operating system requires that the connector (reader) open the connection to the pipe file before the syslog daemon (writer) writes the messages to it. When using Solaris and running the connector as a non-root user, using a Syslog Pipe connector is not recommended. It does not include permissions to send an HUP signal to the syslog daemon.

 Syslog File connectors require syslog configuration to send messages with a certain

syslog facility and severity. For high throughput connectors, Syslog File connectors perform better than Syslog Pipe connectors because of operating system buffer limitations on pipe transmissions.

UNIX supports all three types of syslog connector. If a syslog process is already running, you can end the process or run the connector on a different port.

Because UDP is not a reliable protocol, there is a slight chance of missing syslog messages over the network. TCP is now a supported protocol for syslog connectors.

There is a basic syslog connector, the SmartConnector for UNIX OS Syslog, which provides the base parser for all syslog sub-connectors.

 For syslog connector deployment information, see the SmartConnector Configuration

Guide for UNIX OS Syslog.

 For device-specific configuration information and field mappings, see the SmartConnector Configuration Guide for the specific device. Each syslog sub-connector has its own configuration guide.

During connector installation, for all syslog connectors, choose Syslog Daemon, Syslog Pipe, or Syslog File from the installer selections rather than the name of the syslog sub-connector.

FlexConnectors

ArcSight FlexConnectors let you to create custom connectors that can read and parse information from third-party devices and map that information to ArcSight’s event schema. When creating a custom connector, you define a set of properties (a configuration file) that identify the format of the log file or other source that is imported into the ArcSight ESM Manager or ArcSight Logger.

(28)

Model Import Connectors

Rather than collecting and forwarding events from devices, SmartConnectors for Identity Models import user data from an Identity Management system into ArcSight's IdentityView Solution. See the ArcSight Solution Guide IdentityView for information about how Identity Model Import connectors are used.

ArcSight SmartConnectors for Identity Models extract the user identity information from the database and populate the following lists in ArcSight's IdentityView Solution with the data:

 Identity Roles Session List  Identity Information Session List  Account-to-Identity Map Active List

These lists are populated dynamically, which means that, as the identity data changes in the Identity Manager, the data in the lists is updated when you refresh the session list. Identify Model connectors include:

 SmartConnector for Microsoft Active Directory Identity Model  SmartConnector for Sun Identity Manager Model

 SmartConnector for Oracle IDM Identity Model

Other Connectors

Some connectors use multiple mechanisms. For example, the SmartConnector for Oracle Audit Database monitors both the database tables and audit files. Other examples of connectors with multiple mechanisms include:

Cisco NetFlow File

Retrieves data over TCP in a Cisco-defined binary format. ArcSight Streaming Connector

(29)

Chapter 3 Planning for Deployment

Deployment of an ArcSight SmartConnector is based upon the requirements of your network security enterprise. This section outlines possible ArcSight deployments based upon different scenarios.

The scenarios and deployments shown here are only examples of how you might introduce ArcSight ESM into your enterprise. ArcSight ESM is not limited to just these scenarios and deployments.

Overview

ArcSight components install consistently across UNIX, Windows, and Macintosh platforms. Whether a host is dedicated to the ArcSight ESM Database, Manager, Console, or other component, ArcSight ESM software is installed in a directory tree under a single root directory on each host (DBMS and other third-party software is not necessarily installed under this directory, however.) The path to this root directory is referred to as

$ARCSIGHT_HOME.

In SmartConnector documentation, the 'current' directory is specified rather than presumed to be part of the $ARCSIGHT_HOME location, and the path separator is a backslash (\) (for example, $ARCSIGHT_HOME\current). This is consistent with SmartConnector

configuration guide information, and also underscores the fact that ArcSight

SmartConnectors are not installed on the same machine as the remaining ArcSight ESM components. Rather, they are typically installed on the same machine as the device whose activity will be monitored.

The directory structure below $ARCSIGHT_HOME is standardized across components and platforms. ArcSight software is generally available in the

“Overview” on page 29

“Supported Platforms” on page 30 “Deployment Scenarios” on page 30

“Estimating Storage Requirements” on page 32 “Understanding ArcSight Turbo Modes” on page 33

(30)

ArcSight SmartConnectors collect and process the data generated by various vendor devices throughout your enterprise. Devices consist of routers, email logs, anti-virus products, firewalls, intrusion prevention systems (IPS), access control servers, VPN systems, antiDoS appliances, operating system logs, and other sources where information about security threats are detected and reported.

ArcSight SmartConnectors collect a vast amount of varying, heterogeneous information. SmartConnectors format every raw security event into a consistent, normalized ArcSight event. By creating a consistent message format, you can find, sort, compare, and analyze all events using the same event fields.

When a SmartConnector receives an event, it completes the message by adding device information, then forwarding the event to various components throughout ArcSight ESM.

Supported Platforms

For information about supported platforms, see the ArcSight SmartConnector Product and Platform Support document that is shipped with each SmartConnector release. Only differences to the support detailed in that document are specified in the device's SmartConnector Configuration Guide.

Deployment Scenarios

You can install SmartConnectors on the ArcSight ESM Manager, a host machine, or a device. Based upon configuration, connectors also can receive events over the network using SNMP, HTTP, syslog, proprietary protocols (such as OPSEC), or direct database connections to the device's repository (such as ODBC or proprietary database connections). The best deployment scenario for your system depends upon the SmartConnector type, your network architecture, and your operating system.

 Scenarios for syslog deployment are documented in the SmartConnector for

UNIX OS Syslog Configuration Guide.

 Scenarios for deploying Windows Event Log connectors are documented in the

SmartConnector for Microsoft Windows Event Log Configuration Guide.

Deployment Scenario One

In this scenario, there are three ArcSight SmartConnectors residing on three different devices: a firewall, an IPS, and a UNIX operating system. These connectors receive information from the devices or their logs and send captured events to the ESM Manager based upon the connector configuration.

Once events are received by the Manager, it cross-correlates the events using rules, and sends meta-events to the ESM Database and to any ESM Consoles that access the database.

(31)

3 Planning for Deployment

The ESM Manager also can perform preset actions. Events and meta-events within the ESM Database can be played back using the Replay channel to investigate, analyze, or create a report about event history.

Figure 3-1 Three ArcSight SmartConnectors Residing on Three Devices

Deployment Scenario Two

This scenario is the same as the first, except that the three SmartConnectors reside on a host machine rather than the device itself. The ArcSight SmartConnector need not reside on the device in order to retrieve information from that device. The SmartConnector functions as before, and the ArcSight ESM Manager and Database perform the same functions.

(32)

Deployment Scenario Three

In this scenario, the ArcSight SmartConnectors reside on the ESM Manager itself, not on a host machine, but still retrieve events from devices in the network. The processing performed by the ArcSight SmartConnector, Manager, and Consoles are identical to the other scenarios.

Figure 3-3 Three ArcSight SmartConnectors Residing on an ESM Manager

Estimating Storage Requirements

Understanding the range of devices and SmartConnectors you want to deploy helps in estimating your daily event volume. Log file size is not accurate enough; you need to know how many events are generated during an average day. This varies by the type of device. Not only do different devices generate different event volumes, they also respond differently to various event aggregation policies.

The average size of the data stored for each event depends upon the turbo mode (Fastest, Faster, or Complete) specified for a particular SmartConnector. For detailed information on turbo modes, see the following section, “Understanding ArcSight Turbo Modes”.

SmartConnectors can aggregate events to reduce event traffic. An event that repeats every 500 ms, for example, can be represented by a single event that fires every ten seconds, producing a 20:1 event compression. Individual SmartConnectors can be configured to aggregate events in this manner, reducing event traffic to the ESM Manager and the storage requirements in the Database.

In a distributed environment with multiple ESM Managers, the event volume metric must consider both the SmartConnector feeds to the Manager and the event forwarding from other Managers.

(33)

3 Planning for Deployment

Understanding ArcSight Turbo Modes

You can accelerate the transfer of sensor information through SmartConnectors by choosing one of three turbo modes (Fastest, Faster, or Complete).

The Fastest mode requires the fewest bytes and is most suited to devices such as firewalls, which have relatively little event data. The Faster mode is the Manager default, and requires less storage space. Rich event data sources, such as a network operating system, might use Complete mode, the SmartConnector default. The Complete mode passes all the data arriving from the device, including any custom or vendor-specific (for example, "additional") data.

You can configure SmartConnectors to send more or less event data on a

per-SmartConnector basis, and the ESM Manager can be set to read and maintain more or less event data, independent of the SmartConnector setting.

Some events require more data than others. For example, operating system syslogs often capture a considerable amount of environmental data that may not be relevant to a particular security event. Firewalls, on the other hand, typically report only basic information.

ArcSight defines turbo modes as follows:

When a turbo mode is not specified, Mode 3, Complete, is the default. Versions of ArcSight ESM prior to v3.0 run in turbo mode Complete.

The ESM Manager uses its own turbo mode setting when processing event data. If a SmartConnector is set at a higher turbo mode than the Manager, it reports more event data than the Manager requires. The Manager ignores these extra fields.

However, if a Manager is set at a higher turbo mode than the SmartConnector, the SmartConnector has less event data to report to the Manager. The Manager maintains fields that remain empty of event data.

Both situations are normal in real-world scenarios because the Manager configuration must reflect the requirements of a diverse set of SmartConnectors.

Mode Description

Fastest (Mode 1) Recommended for simpler devices, such as firewalls. Faster (Mode 2) ESM Manager default. Eliminates all but a core set of event

attributes to achieve the best throughput. Because the event data is smaller, it requires less storage space and provides the best performance.

Complete (Mode 3) SmartConnector default. All event data arriving at the SmartConnector, including additional data, is maintained.

(34)

(35)

Chapter 4 Installing SmartConnectors

When you have purchased and are ready to install an ArcSight SmartConnector, see the individual connector’s configuration guide for information specific to the device the connector is monitoring. (For example, when installing a SmartConnector for Windows Event Log, see the SmartConnector Configuration Guide for Microsoft Windows Event Log.) Individual configuration guides contain installation parameter values to enter, how to configure the particular device to enable SmartConnector event collection, and customized device event mappings to ArcSight ESM fields.

Installing ArcSight ESM

Before you install any ArcSight SmartConnectors, make sure that ArcSight ESM has already been installed correctly. Also, ArcSight recommends reading the ArcSight Installation and Configuration Guide before attempting to install a new ArcSight SmartConnector. For a successful installation of ArcSight ESM, follow this order:

1 Ensure that the ArcSight ESM Manager, Database, and Console are installed correctly. 2 Run the ArcSight ESM Manager. The command prompt window or terminal box

displays a "Ready" message when the ESM Manager has started successfully. If the ArcSight ESM Manager is running as a Windows Service or Unix Daemon Service, monitor the server.std.log file located in

$ARCSIGHT_HOME\current\logs\default.

3 Run the ArcSight ESM Console. Although not required, it is helpful to have the Console running when installing the SmartConnector to verify successful installation.

“Installing ArcSight ESM” on page 35 “Installing the SmartConnector” on page 36 “Upgrading SmartConnectors” on page 48 “Running SmartConnectors” on page 50 “Uninstalling a SmartConnector” on page 52

(36)

Before installing the SmartConnector, be sure the following are available:

 Local access to the machine where the SmartConnector is to be installed  Administrator privilege

Installing the SmartConnector

For information regarding operating systems and platforms supported, see the SmartConnector Product and Platform Support document. For complete installation instructions for a particular SmartConnector, see the configuration guide for that connector. The product-specific configuration guide provides device configuration information, installation parameters, and device event mappings to ArcSight ESM fields.

1 Insert the ArcSight Installation CD into your CD-ROM drive or navigate to the location of the ArcSight SmartConnector Installer directory.

2 Start the ArcSight SmartConnector Installer by executing the file for your operating system. Installation files follow the format:

Verify that the ESM Database, Manager, and Console are installed and operating. When the Introduction window is displayed, read the information and click Next when ready.

At a minimum, SmartConnectors must be running version 4021 to communicate with a version 4.0 Manager.

Windows ArcSight-5.0.x.nnnn.y-Connector-Win.exe Solaris ArcSight-5.0.x.nnnn.y-Connector-Solaris.bin AIX ArcSight-5.0.x.nnnn.y-Connector-AIX.bin Linux ArcSight-5.0.x.nnnn.y-Connector-Linux.bin

(37)

4 Installing SmartConnectors

3 Next, accept the default location for "Where Would You Like to Install?," or click Choose… to select another folder for installation. Click Next when ready.

It is a good practice to develop and use a standard naming convention to specify directory locations, file names, and menu option names for the SmartConnectors you install. Typically, if you install multiple connectors on a particular machine, you should install each SmartConnector in a separate directory.

4 Choose from the following types installation; for most connectors, Typical is the appropriate selection. Click Next.

5 On the following window, accept the default shortcut folder location or select a new or existing Program Group. (Windows users can select the Create Icons for All Users

(38)

check box to create icons for all users accessing ArcSight SmartConnectors.) Click Next when you have finished making your selections.

6 Verify your selections on the Pre-Installation Summary window; click Install to begin installation of the SmartConnector core component software.

If the summary is incorrect, click Previous to make changes.

7 An installation process window is displayed during installation of core connector software (click Cancel if you want to cancel the installation).

(39)

8 When the installation of ArcSight SmartConnector core component software is finished, the following window is displayed:

9 Make sure ArcSight Manager (encrypted) is selected and click Next. For information about the ArcSight Logger SmartMessage (encrypted)

destination, see Chapter 8‚ Using SmartConnectors with ArcSight Logger‚ on page 105. For information about NSP Device Poll Listener, see Chapter 9‚ Using

SmartConnectors with NSP‚ on page 117.

10 The Wizard first prompts you for Manager certificate information. The default selection is No, the ArcSight Manager is not using a demo certificate. Choose Yes if ArcSight Manager is using a demo certificate. (Before selecting this option, make sure the Manager is, in fact, using a demo SSL certificate. If you are not certain, select No or consult your system administrator.). If your ArcSight Manager is using a self-signed or CA-signed SSL certificate, select No, the ArcSight Manager is not using a demo certificate and click Next.

11 On the next window, replace localhost with the host name of the Manager with which the SmartConnector is to communicate (localhost is appropriate only when the SmartConnector is installed on the same host as the Manager, which is not recommended in a production environment). This name must match the host name in

After completing the SmartConnector installation wizard, remember to manually configure the connector for the type of SSL certificate your Manager is using. See the ArcSight ESM Administrator's Guide for instructions about configuring your SmartConnector when the Manager is using a self-signed or CA-signed certificate and for instructions about enabling SSL client

authentication on SmartConnectors so that the Connectors and the Manager authenticate each other before sending data.

(40)

the Manager’s certificate, which is usually the fully-qualified name. For example, instead of gabriel, specify gabriel.sales.mycompany.com.

For Manager Port, leave the default value of 8443.

For AUP Master Destination, generally leave this false. If, however, you will have one or more non-ESM destinations, and you want to share this ESM destination's AUP configuration (such as zones) with those destinations, select true. Only do so for one primary destination; if you select true for more than one primary destination or any failover destination, the setting is ignored for all but the first such primary destination. For Filter Out All Events, select true if you want all events filtered out. This means the connector sends no events to this destination. This is useful when an ESM destination is added solely for the purpose of being the AUP master; this value is usually false unless the AUP Master Destination parameter is set to true.

12 Enter a valid ArcSight user name and password for the ArcSight ESM Manager. This is the same user name and password you created during ESM Manager installation.

13 Select one of the possible SmartConnectors from the window displayed. Scroll down to find the appropriate SmartConnector.

(41)

If you are installing a syslog SmartConnector, there are 3 different syslog types: the Syslog Pipe, Syslog File, or Syslog Daemon. For detailed information about syslog SmartConnectors refer to the SmartConnector Configuration Guide for your device. The SmartConnectors that appear in the list are those that can be installed on the same platform from which you are running the installation program. For example, if you are running on Windows, the list contains a list of those SmartConnectors that are supported on Windows. Similarly, if you are running the installer on a Linux or Solaris-based system, the installer displays a list of SmartConnectors supported on those platforms.

14 After selecting the connector you want to install from the list of SmartConnectors, in this example, SAP Security Audit File, click Next.

15 The next window requests specific parameters for the particular SmartConnector you selected. These parameters vary depending upon the device and are described and explained in the SmartConnector Configuration Guide for the selected SmartConnector.

(42)

manually or import multiple hosts. See “Entering Table Parameter Values During Installation” on page 52 for detailed information.

To manually enter parameter values, click the Add button. See “Manually Entering Table Parameter Values” on page 53 for details.

To locate the .csv file you want to import, click the Import button. Click the Export button to create a .csv file containing the values you have entered in the parameter table. See “Importing and Exporting CSV Files” on page 53 for details.

16 Click Next when you have completed entering data.

17 Give your new SmartConnector a descriptive name to identify it for ArcSight Console users. You also can specify optional location information and add any appropriate comments.

If there are no Import and Export buttons on the parameter entry window for the connector you’ve selected, the parameters are not entered into a table format and this feature does not apply.

(43)

In this context, SmartConnector Location refers to the host where you are installing the SmartConnector as well as where within the resource tree this SmartConnector is listed on the ArcSight Console.

Device Location describes the host on which the IDS, syslog, or other software is running. If the device is physical hardware, the Device Location is particularly useful for specifying, for example, a certain position within a specific rack.

18 Click Next when you have finished entering data. 19 Review the summary of data and click Next.

If you choose to configure the SmartConnector to run as a service, the wizard prompts you for the service’s internal and display names.

Each SmartConnector name should be unique. If two similarly named connectors appear in the same SmartConnector Location, an error occurs.

(44)

20 Most SmartConnectors can be installed as a Windows service (or Linux/UNIX daemon) so that the SmartConnector runs automatically when the host is restarted. If the SmartConnector is not configured as a service, it must be started manually whenever it is not running. Select Yes or No and click Next.

21 If you choose not to run the SmartConnector as a service, a window such as the following is displayed.

22 Click Finish to complete connector configuration.

For some SmartConnectors, a system restart is required before the configuration settings you made can take effect. If a System Restart window is displayed, read the information and initiate the system restart operation.

Save any work on your computer or desktop and shut down any other running applications, including the ArcSight Console, if it is running; then shut down the system.

(45)

Installing SmartConnectors from the Command Line

To install ArcSight SmartConnectors without using the graphical user interface wizard, enter –i console on the command line when you invoke the self-extracting archive. Follow the instructions in the command window.

When the installation has successfully completed, manually run the configuration program by executing runagentsetup.

Installing SmartConnectors in Silent Mode

You can run the ArcSight SmartConnector installation program in silent mode, in which answers to wizard questions are provided by a Properties file. This feature is useful for deploying a large number of identical SmartConnectors.

To use this feature, first install and configure one SmartConnector using the graphical-user interface or the command line. While configuring the first SmartConnector, record its configuration parameters in a Properties file. To install all other SmartConnectors in silent mode, use the Properties file you created to provide configuration information.

To record the configuration of a SmartConnector to a Properties file: 1 Run the SmartConnector Configuration Wizard to extract and install the

SmartConnector core files. When the wizard asks you for ESM Manager information, click Cancel.

2 From a command prompt window (from the ARCSIGHT_HOME\current\bin directory), enter the following command to launch the SmartConnector Configuration Wizard in record mode:

On Unix and Linux: ./runagentsetup.sh –i recorderui On Windows: runagentsetup.bat -i recorderui

ArcSight recommends creating and testing the Properties file on a system other than your in-service, production environment. Recording from such a SmartConnector requires removal.

(46)

3 On the window displayed, enter the Silent Properties File Name to select an existing file. Enter the name of the Installation Target Folder to select a location.

4 Continue through all SmartConnector Configuration Wizard windows. The wizard creates a Properties file using the name and location you specified.

Perform the remaining steps on the system on which you want to install the SmartConnector in silent mode:

5 Copy the Properties file from the other system to your current system, preferably to the same directory where you downloaded the installation file.

6 Open the Properties file in an editor of your choice.

7 Find the USER_INSTALL_DIR property in the file and make sure that the path value is the absolute path to the location where you copied the Properties file on this system.

For example, if you copied the Properties file to C:\Program

Files\ArcSightSmartConnectors, the path value should be as follows: ARCSIGHT_AGENTSETUP_PROPERTIES=C\:\\Program

Files\\ArcSightSmartSmartConnectors\\silent_properties

8 From the same file, repeat these steps for the ARCSIGHT_AGENTSETUP_PROPERTIES property.

The equal (=) and backslash (\) characters must be preceded by a backslash (\).

(47)

9 Find the AgentDetailsPanel.smartConnectorname property in the file and change its value to the name of the SmartConnector you are going to install in silent mode, as shown in the following example:

#====================================================== # Panel 'AgentDetailsPanel'

#====================================================== # Select a name for your SmartConnector and specify location parameters. # # SmartConnector Name SmartConnectorDetailsPanel.smartConnectorname=SF_SmartConnector 1 # Agent Location AgentDetailsPanel.agentlocation=San Francisco # Device Location AgentDetailsPanel.devicelocation=Site_2.2.223 # Comment AgentDetailsPanel.comment= #=============================================== 10 If appropriate, edit the following properties:

AgentDetailsPanel.agentlocation AgentDetailsPanel.devicelocation AgentDetailsPanel.comment

You can edit any property (Manager Information, user credentials) in the Properties file to suit your needs.

11 Save the Properties file.

12 Download the SmartConnector installation file appropriate for your platform. 13 Run the following command to install the new SmartConnector in silent mode: