You can manage the amount of log data by defining log pruning filters. Log pruning is needed, for example, when a rule generates both useful and unnecessary logs. Log pruning gives you the ability to discard newly generated irrelevant logs entries on the Log Server. Only logs can be pruned: alerts and audit entries are never pruned.
It is preferable to adjust log generation options instead of letting log entries be generated and then pruning them, as this wastes system resources in creating and transferring the
unnecessary logs.
Table 11.1 Log Task Types
Log Task Explanation
Export Log Task
Export XML copies the selected log data to a separate file in XML/CSV format.
Export CSV copies the selected log data to a separate file in comma-separated (CSV) format.
Export IPS Recordings as PCAP/Snoop copiesIPS traffic recordings to be viewed in an external application in PCAP or Snoop format.
Archive Log Task Copies the selected log data to the current archive folder in StoneGate log file format. The archive task can be set up to additionally delete data.
Delete Log Task Deletes the selected data from the current log files on the Log Server.
Using Log Management Tools 85 You can define two types of log pruning filters:
•Immediate Discard filters delete log entries immediately as they arrive to the Log Server. The deleted log entries are not displayed in the Management Client.
•Discard Before Storing filters deletes log entries before they are saved. Log entries are shown the Current Events mode in the Logs view before they are deleted (in effect, this option converts Essential or Stored type log entries into Transient log entries).
If Domain elements have been configured, log pruning filters can only be defined in the Shared Domain. The same log pruning filters are used in all Domains. Administrators who have the right to manage log pruning can view the log pruning filters when they are logged in to other Domains.
Using Log Management Tools
About the Log Files
Logs are stored as a files on the Log Server. A separate folder is created for the logged events each hour. The log files are stored by default in the <installation directory>/data/
storage/ directory on the Log Server.
The log files have the following naming: YYYYMMDD_hh_C<ORIGINATOR>_MMDDhhmmsssss.arch.
•The date YYYYMMDD_hh refers to the date and hour of the logged events.
•The rest of the file name starting with “_C” refers to the file creation date and the
<ORIGINATOR> refers to the originator of the logged events.
The time and date in the file name always use the UTC time zone, which is the system’s internal time zone.
Archive Directories
Log files are archived by default in the Log Server’s default archive directory <installation directory>data/archive. You can change this directory and define up to 32 alternative or additional directories. For example, you could directly archive some or all of the logs on a network drive to free resources on the Log Server. See the Online Help of the Management Client and the Administrator’s Guide PDF for more information.
Exporting Log Data to Syslog Servers
Log entries can be exported from a Log Server to external syslog servers. The log entries are selected with log filters. The filters must be exported to a file to be used for syslog filtering. Of log pruning is applied, any entries that pass “immediate discard” pruning can be sent to the syslog servers. Entries do not need to be stored on the Log Server to be sent to the syslog server. See the Online Help of the Management Client and the Administrator’s Guide PDF for more information.
Caution – Be careful when defining the pruning filters. The matching log events are irreversibly deleted at the Log Server.
86 Chapter 11 Log Management
Examples of Log Management
The examples in this section illustrate some common uses for log management in StoneGate and general steps on how each scenario is configured.
Archiving Old Logs
Last month’s logs are taking up too much disk space on the Log Servers at Company A, but some of the logs are still needed for the company’s records. The administrators decide to archive the needed logs on another server and to delete last month’s log data from the Log Servers. Because not all the logs from last month need to be archived, they delete the unnecessary logs altogether. They want to repeat the same archiving operation once a month from now on. To do this the administrators:
1. Create a new Archive Log Task for archiving the data with the following settings:
2. Save the new Archive Log Task.
3. Create a new Scheduled Task for running the Archive Log Task and set it to be repeated monthly.
4. Save the Scheduled Task.
Filtering Out Irrelevant Logs
Illustration 11.2 Company B’s Network
Server A provides services to users in network X, as well as to Server B. The administrators are interested in tracking how many of the users in network X actually use Server A. Server B also connects frequently to Server A, and generates a large amount of traffic.
Table 11.2 Archive Log Task for Company A
Option Setting
Time Range Last Full Month
Filter for Copying A custom filter which matches the important log data that the administrators want to archive.
Filter for Deletion Match All to delete all last month’s logs from the Log Server.
Archive Target
Directory A network drive.
Server A Server B
Host
Network X
Examples of Log Management 87
Since Server B makes very frequent connections to Server A and the administrators are only interested in connections from other hosts in network X to Server A, the administrators decide to temporarily eliminate logs that are a result of Server B’s connections. All hosts in network X including Server B are currently logged according to a single rule. Creating a separate rule to handle just the Server B connections with logging set to “None” would create unnecessary clutter in the policy when the administrators only want to filter the logs from Server B temporarily. The administrators decide to set up log pruning to filter the logs so that only the relevant ones are stored on the Log Server. To do this the administrators:
1. Select one of the irrelevant log entries in the Logs view.
2. Create a temporary filter based on the log entry, and save the filter as a permanent filter.
3. Add the new filter to the Discard Before Storing list in the Log Data Pruning view.
88 Chapter 11 Log Management
89
C HAPTER 12
A LERT E SCALATION
Alert Escalation means defining when and how the system notifies the administrators when an alert entry is created.
The following sections are included:
Overview to Alert Escalation (page 90)
Configuration of Alert Escalation (page 90)
Using Alert Escalation (page 93)
Examples of Alert Escalation (page 96)
90 Chapter 12 Alert Escalation
Overview to Alert Escalation
Alert entries inform the administrators if there is a problem with the StoneGate system, test or task failure, or if a rule of particular interest matches (usually mainly Inspection rules). Alert entries can be viewed in the Logs view in the same way as other logs, but of course it is not wise to rely on administrators noticing critical messages from the general log stream. This is why Alerts have an escalation process that can notify the administrators of new alerts in several different ways.
By default, notifications of new alert entries are shown in the Management Client. You can also configure the Log Server to send alert notifications out from the system through different channels according to various criteria.
Some form of alert escalation is useful in any environment. We recommend you make sure that the system attracts the attention of the administrators to at least those alert entries that indicate problems in the system components (System Alerts).
Configuration of Alert Escalation
Illustration 12.1 Alert Escalation
The alerts are handled as follows:
1. An alert entry is triggered on a system component.
2. The alert entry is sent to the Log Server, which stores it. At this point, administrators can view the alert in the Logs view.
3. The Log Server matches the alert entry to its Alert Policy to select the correct Alert Chain.
4. The Alert Chain triggers a series of notifications.
For example, an Alert Chain can first notify one of the administrators by e-mail, wait for acknowledgement for 10 minutes, and if the alert is not acknowledged in time, the Log Server can notify this or some other administrator with a mobile phone text message. Alert
notifications are sent until the alert is acknowledged by one of the administrators or when the Alert Chain is completed.
Alert escalation is highly configurable, so you can for example:
•Define which alert entries are escalated to which administrators based on who is responsible for which components.
•Configure different alert escalation rules for weekends and evenings.
•Set up alert escalation so that if administrators responsible for, for example, a branch office site do not acknowledge an alert, the alert escalation continues to administrators at a nearby site or at the headquarters.
1 2 3 4
Configuration of Alert Escalation 91
Default Elements
By default, the system includes the following alert-related elements:
•System Alert: this Alert element is used for alerts triggered by critical events in the system’s internal operation (the System Situations explained below).
•Default Alert: this Alert element is used in the default Inspection rules. You can also use it in your own custom configurations.
•Test Alert: this Alert element is used when you test alert handing.
•Default Alert Policy: this Alert Policy contains a rule that escalates all alerts using the Default Alert Chain.
•Default Alert Chain: this Alert Chain escalates all alerts to all administrators using the User Notification channel (alert icon in the Management Client).
•System Situations: the System Situations contain definitions for problems in the system that trigger a System Alert. There are no configurable parameters for System Situations and you cannot adjust when these Situations are triggered. When you configure alert escalation, make sure that your custom Alert Policies escalate the System Alerts as well, because the System Alerts always require administrator action.
Configuration Workflow
The following sections provide an overview of the configuration tasks. Detailed step-by-step instructions can be found in the Online Help of the Management Client and the Administrator’s Guide PDF.