Situations reference - Linux OS Agent User s Guide

This chapter contains an overview of situations, references for detailed information about situations, and descriptions of the predefined situations included in this monitoring agent.

About situations

A situation is a logical expression involving one or more system conditions.

Situations are used to monitor the condition of systems in your network. You can manage situations from the Tivoli Enterprise Portal by using the Situation editor.

The IBM Tivoli Monitoring agents that you use to monitor your system

environment are shipped with a set of predefined situations that you can use as-is or you can create new situations to meet your requirements. Predefined situations contain attributes that check for system conditions common to many enterprises.

Using predefined situations can improve the speed with which you can begin using the Monitoring Agent for Linux OS. You can examine and, if necessary, change the conditions or values being monitored by a predefined situation to those best suited to your enterprise.

Note: The predefined situations provided with this monitoring agent are not read-only. Do not edit these situations and save over them. Software updates will write over any of the changes that you make to these situations.

Instead, clone the situations that you want to change to suit your enterprise.

You can display predefined situations and create your own situations by using the Situation editor. The left frame of the Situation editor initially lists the situations associated with the Navigator item that you selected. When you click a situation name or create a new situation, the right frame opens with the following tabs:

Formula

Condition being tested Distribution

List of managed systems (operating systems, subsystems, or applications) to which the situation can be distributed.

Expert Advice

Comments and instructions to be read in the event workspace Action

Command to be sent to the system Until Duration of the situation

More information about situations

The IBM Tivoli Monitoring User's Guide contains more information about predefined and custom situations and how to use them to respond to alerts.

For a list of the predefined situations for this monitoring agent and a description of each situation, refer to the Predefined situations section below and the

information in that section for each individual situation.

Predefined situations

This monitoring agent contains the following predefined situations:

The remaining sections of this chapter contain descriptions of each of these predefined situations. The situations are organized alphabetically.

v Linux_AMS_Alert_Critical

v Linux_BP_AvgCpuBusyPct1h_Critic v Linux_BP_CpuBusyPct_Critical v Linux_BP_CpuWaitIOPct_Warning v Linux_BP_LoadAvg5min_Critical v Linux_BP_NetTotalErrPct_Warning v Linux_BP_NumberZombies_Warning v Linux_BP_ProcHighCpu_Critical v Linux_BP_ProcMissing_Critical v Linux_BP_SpaceUsedPct_Critical v Linux_BP_SpaceUsedPctCustom_Cri v Linux_BP_SwapSpaceUsedPct_Criti v Linux_Fragmented_File_System v Linux_Fragmented_File_System_2 v Linux_High_CPU_Overload v Linux_High_CPU_Overload_2 v Linux_High_CPU_System v Linux_High_CPU_System_2 v Linux_High_Packet_Collisions v Linux_High_Packet_Collisions_2 v Linux_High_RPC_Retransmit v Linux_High_RPC_Retransmit_2 v Linux_High_Zombies

v Linux_High_Zombies_2 v Linux_Low_Pct_Inodes v Linux_Low_Pct_Inodes_2 v Linux_Low_percent_space v Linux_Low_percent_space_2 v Linux_Low_Space_Available v Linux_Low_Space_Available_2 v Linux_Network_Status

v Linux_Network_Status_2 v Linux_NFS_Buffer_High v Linux_NFS_Buffer_High_2 v Linux_NFS_Getattr_High v Linux_NFS_Getattr_High_2 v Linux_NFS_rdlink_high v Linux_NFS_rdlink_high_2 v Linux_NFS_Read_High v Linux_NFS_Read_High_2

v Linux_NFS_Writes_High v Linux_NFS_Writes_High_2 v Linux_Packets_Error v Linux_Packets_Error_2 v Linux_Process_High_Cpu v Linux_Process_High_Cpu_2 v Linux_Process_High_Instant_CPU v Linux_Process_stopped

v Linux_Process_stopped_2 v Linux_RPC_Bad_Calls v Linux_RPC_Bad_Calls_2 v Linux_System_Thrashing v Linux_System_Thrashing_2

Linux_AMS_Alert_Critical situation

Monitors to determine if one of the following conditions is true:

v A managed agent has exceeded its restart count for the day as configured in the 'maxRestarts' field of its Common Agent Package file.

v A managed agent is overutilizing the available CPU resources as configured in the 'cpuThreshold' field of its Common Agent Package file.

v A managed agent is overutilizing the available system memory resources as configured in the 'memoryThreshold' field of its Common Agent Package file.

v An attempt at auto-restarting a managed agent failed.

v An attempt at starting a stopped or manually stopped managed agent failed.

v The Agent Management Services watchdog is no longer reliable. If either watchdog stops monitoring, you will receive this message.

The formula for this situation is as follows:

Alert Message==’Agent exceeded restart count’ OR Alert Message==’Agent overutilizing CPU’ OR Alert Message==’Agent overutilizing memory’ OR Alert Message==’Agent restart failed’ OR Alert Message==’Agent manual stop failed’ OR

Alert Message ==’Agent Management Services watchdog no longer reliable’

Linux_BP_AvgCpuBusyPct1h_Critic situation

Monitors total percent CPU (system + user) busy for the sum of all the CPU on the system. This situation is automatically installed, but not automatically distributed, during installation to the default MSL or managed system.

The formula for this situation is as follows:

*IF *VALUE KLZ_CPU_Averages.CPU_Usage_Current_Average *GE 95.0

Linux_BP_CpuBusyPct_Critical situation

Monitors if the CPU workload is high. This situation is automatically installed, but not automatically distributed, during installation to the default MSL or managed system.

The formula for this situation is as follows:

*IF ( ( *VALUE KLZ_CPU.CPU_ID *NE Aggregate *AND *VALUE KLZ_CPU.Busy _CPU *GT 95.0 ) *OR ( *VALUE KLZ_CPU.CPU_ID *EQ Aggregate *AND *VALUE KL Z_CPU.Busy_CPU *GT 90.0 ) )

Linux_BP_CpuWaitIOPct_Warning situation

Monitors CPU I/O waits. This situation is automatically installed, but not automatically distributed, during installation to the default MSL or managed system.

The formula for this situation is as follows:

*IF *VALUE KLZ_CPU.Wait_IO_CPU *GT 10 *AND *VALUE KLZ_CPU.CPU_ID *EQ Aggregate

Linux_BP_LoadAvg5min_Critical situation

Monitors the load on the system for the last five minutes. This situation is

automatically installed, but not automatically distributed, during installation to the default MSL or managed system.

The formula for this situation is as follows:

*IF ( ( *VALUE Linux_Machine_Information.Number_of_Processors_Online

*EQ 1 *AND *VALUE KLZ_System_Statistics.System_Load_5min *GT 4.0 ) *OR ( *VALUE Linux_Machine_Information.Number_of_Processors_Online *EQ 2 *AN D *VALUE KLZ_System_Statistics.System_Load_5min *GT 8 ) *OR ( *VALUE Lin ux_Machine_Information.Number_of_Processors_Online *EQ 3 *AND *VALUE KLZ _System_Statistics.System_Load_5min *GT 12 ) *OR ( *VALUE Linux_Machine_

Information.Number_of_Processors_Online *EQ 4 *AND *VALUE KLZ_System_Sta tistics.System_Load_5min *GT 16 ) *OR ( *VALUE Linux_Machine_Information .Number_of_Processors_Online *EQ 6 *AND *VALUE KLZ_System_Statistics.Sys tem_Load_5min *GT 24 ) *OR ( *VALUE Linux_Machine_Information.Number_of_

Processors_Online *EQ 8 *AND *VALUE KLZ_System_Statistics.System_Load_5m in *GT 32 ) )

Linux_BP_NetTotalErrPct_Warning situation

Monitors the percentage of errors of received or transmitted network packets. This situation is automatically installed, but not automatically distributed, during installation to the default MSL or managed system.

The formula for this situation is as follows:

*IF *VALUE KLZ_Network.Total_Error_Percent *GT 10

Linux_BP_NumberZombies_Warning situation

Monitors the number of processes for defunct. This situation is automatically installed, but not automatically distributed, during installation to the default MSL or managed system.

The formula for this situation is as follows:

*IF *VALUE KLZ_System_Statistics.Processes_Zombie_Count *GE 50

Linux_BP_ProcHighCpu_Critical situation

Monitors the CPU percent utilization by all processes except kswapd. This situation is automatically installed, but not automatically distributed, during installation to the default MSL or managed system.

The formula for this situation is as follows:

*IF *VALUE KLZ_Process.Busy_CPU_Pct *GE 95.0 *AND *VALUE KLZ_Process .Process_Command_Name *NE kswapd

Linux_BP_ProcMissing_Critical situation

Monitors specified processes that are not found in the system. The process might have been killed or might never have been started. An asterisk is used to identify the specific process started from the system directory. This situation is

automatically installed, but not automatically distributed, during installation to the default MSL or managed system.

The formula for this situation is as follows:

*IF *MISSING KLZ_Process.Proc_CMD_Line *EQ ( ’/usr/*/sshd’,’/usr/*/s yslogd’,’/usr/*/ntpd’,’/usr/*/inetd’,’/usr/*/crond’ )

Linux_BP_SpaceUsedPct_Critical situation

Monitors all mounted file systems for space used percentage. This situation is automatically installed, but not automatically distributed, during installation to the default MSL or managed system.

The formula for this situation is as follows:

*IF *VALUE KLZ_Disk.Disk_Used_Percent *GE 95 *AND *VALUE KLZ_Disk.FS _Type *NE nfs

Linux_BP_SpaceUsedPctCustom_Cri situation

Monitors only specific file systems for space used percentage, for example, /home.

This situation is automatically installed, but not automatically distributed, during installation to the default MSL or managed system.

The formula for this situation is as follows:

*IF ( ( *VALUE KLZ_Disk.Mount_Point *EQ ’/’ *AND *VALUE KLZ_Disk.Dis k_Used_Percent *GE 90 ) *OR ( *VALUE KLZ_Disk.Mount_Point *EQ ’/home’ *A ND *VALUE KLZ_Disk.Disk_Used_Percent *GE 80 ) ) *UNTIL ( *SIT Linux_BP_S paceUsedPct_Critical )

Linux_BP_SwapSpaceUsedPct_Criti situation

Monitors the percentage of swap space used. This situation is automatically installed, but not automatically distributed, during installation to the default MSL or managed system.

The formula for this situation is as follows:

*IF *VALUE KLZ_VM_Stats.Swap_Used_Pct *GT 40

Linux_Fragmented_File_System situation

This situation has been superseded by Linux_Fragmented_File_System_2. Monitors the percentage of i-nodes to disk space. An exception condition occurs when the percentage of i-nodes to disk space used is high, which could indicate high disk fragmentation on the disk.