IBM Tivoli Monitoring provides several tools; some include functionality for diagnosing problems. The primary diagnostic tool is logging. Logging refers to the text messages and trace data generated by the software. Messages and trace data are sent to an output destination, such as a console screen or a file.
Trace logging
Trace logs capture information about the operating environment when component software fails to operate as intended. IBM Software Support uses the information captured by trace logs to trace a problem to its source or to determine why an error occurred.
Trace logs are commonly referred to asRAS1logs because RAS1 is the name of the IBM Tivoli Monitoring component that manages trace logging. Furthermore, the two tracing-related environment variables areKBB_RAS1, which sets a product's tracing level, andKBB_RAS1_LOG, which assigns the name, size, count, and directory location of RAS1 log files.
By default, RAS1 logs on distributed systems are stored in the/logs directory under the installation path for IBM Tivoli Monitoring. On z/OS, the RAS1 log is stored as a SYSOUTfile associated with theRKLVLOG DDNAME.
When an IBM Tivoli Monitoring product is installed and configured, RAS1 tracing for the product is configured by default to specify ERROR-level logging, which means only the most important run-time messages are traced. There are different methods for customizing the logging level, along with the size, count, and location of log files.
Log file locations
Log files are saved in log and component directories in your IBM Tivoli Monitoring installation.
The mechanism for managing log files is the KBB_RAS1_LOG environment variable.
KBB_RAS1_LOG environment variable
The KBB_RAS1_LOG environment variable specifies the full path name of a product's log file, the full path name of the inventory control file, and several options for controlling logging behavior.
Unless instructed to do so by IBM Software Support, you should not normally modify the KBB_RAS1_LOG default values because of the risk that critical logging information could be lost.
The KBB_RAS1_LOG has the following format: KBB_RAS1_LOG=filename [setting=value]
[COUNT=count]
Maximum number of log files to create in one invocation of the product. The default count is 5. Each log file created in a product invocation gets assigned a
count, starting at 01, that is stored in the log file name. Whenever a log file fills and a new log is created, the count gets incremented by 1, up to the limit defined in the count parameter. The new log sequence number is stored as an
nnvalue in the log file name, for example, the03insystema_ms_4f2b12eb- 03.log.
[INVENTORY=inventory_filename]
A file, with a .invextension on distributed platforms, which automatically records the history of log files across the most recent invocations of the product. By default, the inventory control file is located in theinstall_dir/logs directory. The name of the file includes the local system name and the two-character product code, for example, WINSYS1_cq.inv or
AIXPROD_ux.inv. If you cannot find RAS1 logs that you are searching for, you should examine the product's inventory control file and review the names and locations of the log files listed there.
[LIMIT=limit]
Maximum size per log file. The default size is 8 megabytes.
[MAXFILES=maxfiles]
The total number of log files in a product's inventory that will be saved. Note that saved log files can span across multiple product invocations, provided that the maxfiles limit has not yet been reached. Extra log files beyond the maxfiles value are automatically deleted. There must be a valid inventory control file in order for the maxfiles value to be enforced.
[PRESERVE=preserve]
The number of log files to preserve when log files exceed the count. The default is 1. This means that if, for example, the log file count is 5 and all 5 logs are filled while the product is running, the 6th log will overwrite the -02.log, the 7th log will overwrite the -03 log, and so forth. But the -01 log will be preserved, which is important because it contains a product's startup messages and valuable configuration information. Unless instructed to do so by IBM Software Support, you should not normally modify the
KBB_RAS1_LOG default values because of the risk that critical logging information could be lost.
Log file naming
RAS1 log files are maintained with the following naming, as determined by the KBB_RAS1_LOG environment variable in the product's configuration file:
v RAS1 logs are stored in the\logs directory under the installation path for
IBM Tivoli Monitoring. The following is an example of a log file name that includes the local system name, the two-character product code of the Tivoli Enterprise Portal Server, the time stamp in hexadecimal format when the process started, and the log sequence number:
ibm-kpmn803v01_cq_472649ef-02.log
v On UNIX-based systems, RAS1 logs are stored in the/logsdirectory
under the installation path for IBM Tivoli Monitoring. The following example of a log file name includes the local system name, the two-character product code of the UNIX OS Monitoring Agent, the agent's “stat_daemon” child process name, the time stamp in hexadecimal format when the process started, and the log sequence number:
f50pa2b_ux_stat_daemon_49ac1eee-01.log
Note: When you communicate with IBM Software Support, you must capture and send the RAS1 log that matches any problem occurrence that you report.
Log file path
Here are the location of the trace log files that are associated with the use of the following components. The default paths forinstall_dir areC:\Program Files\IBM
on Windows and/opt/ibm/on Linux of UNIX.
Tivoli Enterprise Portal Server
install_dir\logs
install_dir/logs/ hostname_CQ_timestamp.log
where:
install_dir specifies the directory where the portal server was installed.
hostnamespecifies the name of the system hosting the product.
CQis the component code for the portal server.
timestamp is a decimal representation of the time when the process was started.
Tivoli Enterprise Portal browser client and Java Web Start
Location of trace log files that are associated with the use of the Tivoli Enterprise Portal when the client is deployed within a browser or as a Java Web Start application:
The log location depends somewhat on the version of Windows being used. Here are the two most common locations:
%USERPROFILE%\AppData\LocalLow\IBM\Java\Deployment\log
%USERPROFILE%\Application Data\IBM\Java\Deployment\log
${user.home}/.java/deployment/log
For the browser client, the file name ispluginnnnnn.trace. For Java Web Start, the file name isjavawsnnnnn.trace.
wherennnnnis a unique, randomly generated numeric suffix to support generational logs.
Tivoli Enterprise Portal desktop client
install_dir\cnp\logs\kcjerror_n.log
install_dir\cnp\logs\kcjras1.log
install_dir\cnp\kcj.log
install_dir/logs/hostname_CJ_timestamp.log
where:
install_dir specifies the directory where the portal client was installed.
_nrepresents the circular sequence in which logs are rotated. The logs range from no _nfor the current log file, to 1 through 9 for the previous logs. A newkcjerror.logfile is generated each time the desktop client is started, at which time the previous log is renamed kcjerror_1.log. If there was akcjerror_1.log, that one gets renamed tokcjerror_2.log, and so on until 9 is reached and the logs start over with kcjerror_1.log.
hostnamespecifies the name of the system hosting the product.
CJis the component code for the portal client.
timestamp is a decimal representation of the time when the process was started.
You can configure for multiple named instances of the desktop client. This is typically done, for example, when you want to have multiple desktop client instances connecting to different portal server environments. When you use the “Create instance” action from either the Windows Manage Tivoli Enterprise Monitoring Services utility or the Linux CandleManage panel associated with the desktop client, you are prompted to provide a name for the new instance. The default instance has no name. All the log file names for the desktop client include the instance name, so the file naming conventions for the 3 logs are as follows: kcjinstance_name.log,
kcjerrorinstance_name.log, andkcjras1instance_name.log. For Linux, the
kcjinstance_name.logfile actually uses the standard Linux log filename convention ofhostname_CJ_timestamp.log.
Tivoli Enterprise Monitoring Server
install_dir\logs\hostnameMS_ HEXtimestamp-nn.log
install_dir/logs/hostname_MS_timestamp.log
where:
install_dir specifies the directory where the monitoring server was installed.
hostnamespecifies the name of the system hosting the monitoring server.
MSis the component code for the monitoring server.
HEXtimestampis a hexadecimal representation of the time when the process was started.
nnrepresents the circular sequence in which logs are rotated. The logs range from 1 to 5, by default, although the first is always retained because it includes configuration parameters.
Dashboard Application Services Hub
C:\Program Files\IBM\JazzSM\profile\logs\server1\SystemOut.log /opt/ibm/JazzSM/profile/logs/server1/SystemOut.log
tivcmd Command-Line Interface for Authorization Policy
C:\IBM\TivoliMonitoring\logs\kdqras1_51229cb8-01.log
/opt/IBM/TivoliMonitoring/logs/kdqras1_51229cb8-01.log
Audit logs for the Authorization Policy Server
C:\Program Files\IBM\JazzSM\AuthPolicyServer\PolicyServer\ audit\hostname_2013.02.18_16.04.38.458.w7_audit.log
/opt/IBM/JazzSM/AuthPolicyServer/PolicyServer/audit/
hostname_2013.02.18_16.04.38.458.w7_audit.log
where:
hostnameis the name of the computer where the Authorization Policy Server is installed.
2013.02.18_16.04.38.458.w7is the log time stamp.
Automation Server
install_dir\logs\kasmain.msg
install_dir/logs/hostname_AS_HEXtimestamp-n.log
hostnameis the name of the computer where the automation server is installed.
ASis the component code for the Automation Server.
HEXtimestampis the log time stamp in hexadecimal.
n is the log number.
Monitoring agents
install_dir\tmaitm6\logs\ hostname_PC_HEXtimestamp-nn.log
install_dir/logs/hostname_PC_timestamp.log
where:
install_dir specifies the directory where the monitoring agent was installed.
hostnamespecifies the name of the system hosting the monitoring agent.
PCspecifies the product code, for example, NTfor Windows OS.
HEXtimestampis a hexadecimal representation of the time when the process was started.
nnrepresents the circular sequence in which logs are rotated. The logs range from 1 to 5, by default, although the first is always retained because it includes configuration parameters.
IBM Tivoli Warehouse Proxy agent
install_dir\logs\hostname_HD_ timestamp.log
install_dir/logs/hostname_PC_ timestamp.log
where:
install_dir specifies the directory where the monitoring agent was installed.
hostnamespecifies the name of the system hosting the Warehouse Proxy agent.
HDis the product code for the IBM Tivoli Warehouse Proxy agent.
IBM Tivoli Summarization and Pruning agent
The Summarization and Pruning Agent uses C-based RAS1 tracing, Java-based RAS1 tracing and Java-based internal tracing. By default, Summarization and Pruning Agent trace data is written to a file in the logs subdirectory.
install_dir\logs\hostname_SY_ HEXtimestamp-nn.log
install_dir\logs\hos tname_SY_ ras1java_HEXtimestamp-nn.log
install_dir\logs\hostname_PC_ java_HEXtimestamp-nn.log
install_dir/logs/hostname_SY_ HEXtimestamp-nn.log
install_dir/logs/hostname_SY_ras1java_ HEXtimestamp-nn.log
install_dir/logs/hostname_SY_java_ HEXtimestamp-nn.log
where:
install_dir specifies the directory where the monitoring agent was installed.
hostnamespecifies the name of the system hosting the monitoring agent.
HEXtimestampis a hexadecimal representation of the time when the process was started.
nnrepresents the circular sequence in which logs are rotated. The logs range from 1 to 5, by default, although the first is always retained because it includes configuration parameters.
Installation log files
Use the log files that are created during installation to help diagnose any errors or operational issues.
The following table lists and describes the log files created when installing when installing a Tivoli Enterprise Monitoring Server, Tivoli Enterprise Portal Server, Tivoli Enterprise Portal client, and Tivoli Enterprise Monitoring Agent:
Table 1. Installation log files
Windows UNIX-based systems
v ITM_HOME\InstallITM\Abort<Product_name><date_timestamp>.log
This log is created if an abort occurs for either a first time
installation or a modification of previous installation of IBM Tivoli Monitoring.
v
ITM_HOME\InstallITM\<Product_name>_<timestamp>.log
This log is created during a normal clean installation.
v
ITM_HOME\InstallITM\MOD_<Product_name>timestamp.log
This log is created if you modify an existing product specified with the PC, or when adding or deleting components.
where:
Product_name
Specifies the product name. IBM Tivoli Monitoring 20050923 1815.logis the log file name for the IBM Tivoli Monitoring installation CD.
timestamp
A decimal representation of the time at which the process was started.
$CANDLEHOME/logs/candle_ installation.log
You can find a log for uninstallation on Windows in the root directory where the product was installed:
Uninstall<PC><date_timestamp>.log
Windows installer and configuration logs
Obtain details about the installation (or upgrade) process in the logging and tracing information. You can set the trace levels.
You can set the degree of logging and tracing to one of three levels:
v DEBUG_MIN v DEBUG_MID v DEBUG_MAX
By default, logging and tracing is set to DEBUG_MIN. Higher levels give you more detailed information about the installation process. This can be useful for investigating any problems or errors that occur.
Level name What is logged or traced
DEBUG_MIN Most important method entries, exits and trace messages are traced
DEBUG_MID Most of the method entries, exits and trace messages are traced DEBUG_MAX All of the method entries, exits and trace messages are traced
You can set the level of logging and tracing by using the/zflag when you execute thesetup.exe file in the CLI.
v For GUI installation use one of the following commands:
– setup.exe /zDEBUG_MAX
– setup.exe /zDEBUG_MID
– setup.exe /zDEBUG_MIN
v For silent installation use one of the following commands:
– start /wait setup /z"DEBUG_MAX/sfC:\temp\SILENT_SERVER.txt" /s /f2"C:\temp\silent_setup.log"
– start /wait setup /z"DEBUG_MID/sfC:\temp\SILENT_SERVER.txt" /s /f2"C:\temp\silent_setup.log"
– start /wait setup /z"DEBUG_MIN/sfC:\temp\SILENT_SERVER.txt" /s /f2"C:\temp\silent_setup.log"
UNIX installer and configuration logs
Obtain details about the installation (or upgrade) process in the logging and tracing information. You can set the trace levels.
For tracing and logging Java code (that is run on UNIX systems), this mechanism enables problem debugging. Two sets of information are created – logs and traces. Logs (*.log) are globalized and traces (*.trc) are in English. They contain entry and exit parameters of method and stack traces for exceptions. The amount of
information traced depends on the level of tracing set.
Level name What is logged or traced
LOG_ERR Only exceptions and errors are logged and traced
LOG_INFO Also log messages are logged and traced - DEFAULT
DEBUG_MIN Also most important method entries, exits and trace messages are traced
DEBUG_MID Most of the method entries, exits and trace messages are traced
DEBUG_MAX All of the method entries, exits and trace messages are traced
The level can be set in configuration files or by exporting an environment variable called TRACE_LEVEL with one of the values mentioned above. Configuration of RAS settings is stored in the following files:
v CH/config/ITMConfigRAS.properties(for configuration)
Callpoints are the only component that is handled differently, their logs and traces always go to the directoryCH/InstallITM/plugin/executionEvents. The default location for installation isCH/logs/itm_install.log(.trc) and for configuration it isCH/logs/itm_config.log(.trc).
To gather all the needed logs and environment information in case of an error, use the pdcollect tool. See “pdcollect tool” on page 68.
Component Location File name
Install logs/traces CH/logs
candle_installation.log itm_install.log (.trc) Config logs/traces CH/logs itm_config.log (.trc) Logs for component startup CH/logs
pc.env (lists env variables passed to the agent) hostname_pc_ID.log Callpoint logs/traces CH/InstallITM/plugin/
executionEvents/logs/ timestamp/install(config)/ plugin_type/pc callpoint.trc (.log) *.stderr *.stdout
Tivoli Distributed Monitoring upgrade log file
All upgrade actions performed by the IBM Tivoli Monitoring Upgrade Toolkit are recorded in a central log with an associated user ID and a time stamp.
Upgrade actions taken outside of the Upgrade Toolkit are not recorded in the log.
Table 2. Upgrading from Tivoli Distributed MonitoringTivoli log file
Windows UNIX-based systems
$DBDIR/AMX/logs/log_tool_ timestamp.log $DBDIR/AMX/logs/log_tool_ timestamp.log where: $DBDIR
The Tivoli Management Environment Framework environment variable that specifies the directory where the Object Repository (odb.bdb) is located.
tool Specifies the IBM Tivoli Monitoring Upgrade Toolkit tool: witmscantmr, witmassess, or witmupgrade.
timestamp
Specifies a time stamp that includes data and time of execution. For example: log_witmscantmr_20050721_15_30_15.log
The log file name displays when the Upgrade Toolkit tool completes the upgrade operation. Each time a Upgrade Toolkit tool runs, its generates a new log file that is never reused by any tool. The contents of the log file conform to the Tivoli Message Standard XML logging format. The following example is an excerpt from
<Message Id="AMXUT2504I" Severity="INFO">
<Time Millis="1121977824199"> 2005.07.21 15:30:24.199 CST </Time> <Server Format="IP">YFELDMA1.austin.ibm.com</Server>
<ProductId>AMXAMX</ProductId> <Component>ScanTMR</Component> </Component>1</ProductInstance>
<LogText><![CDATA[AMXUT2504I The software is creating a new baseline file C:\PROGRA~1\Tivoli\db\YFELDMA1.db\AMX\shared\analyze\scans\ 1889259234.xml.]]; </LogText> <TranslationInfo Type="JAVA" Catalog="com.ibm.opmt.utils.messages.MigrationManager_ msgs" MsgKey="AMXUT2504I"><Param> <![CDATA[C:\PROGRA~1\Tivoli\db\YFELDMA1.db\AMX\shared\analyze\scans\ 1889259234.xml]]; </Parm></TranslationInfo> <Principal></Principal> </Message>
Reading RAS1 logs
RAS1 logs are primarily a diagnostic tool for IBM Software Support. However, the logs can also be read by administrators to gain an understanding of the major events in the life of an IBM Tivoli Monitoring process.
Even with default ERROR-level logging, you can find information in RAS1 logs about product configuration, security settings, network interfaces, listening ports, key milestones during startup and shutdown, run-time errors, user logins, commands issued, etc.