Chapter 4. Business Service Management sample implementation
4.7 Defining TEC rules
4.7.9 Defining TBSM operators
7. For historical data collection, we need to set up Tivoli Data Warehouse to collect data from the monitoring application. See 4.8, “Configuring Tivoli Data Warehouse” on page 151.
8. For Service Level monitoring, we need to set up IBM Tivoli Service Level Advisor offerings, customers and orders. See 4.9.1, “Defining the operation”
on page 160.
The installed software shown in Table 4-3 is a subset of the overall software configuration shown in Table 3-9 on page 64.
Table 4-3 Software configuration list
Machine name Operating System Software list IBMTIV1
TMR Server Gateway RIM host
AIX 5L Tivoli Management Framework 4.1 IBM Tivoli Monitoring 5.1.1
IBM Tivoli Monitoring Component Services 5.1 IBM Tivoli Monitoring for *
DB2 Universal Database Version 7.1 IBM Tivoli Enterprise Console 3.8
IBM Tivoli Business Systems Manager distributed edition 2.1.1
IBM Tivoli Monitoring for Transaction Performance:
Web Service Courier endpoint IBMTIV5 Windows 2000 Server Tivoli Management Framework 4.1
IBM Tivoli NetView 7.1.3 BSM9
TIMS
Windows 2000 Server IBM Tivoli Monitoring for Transaction Performance 5.2:
Internet Management Server BSM4, BSM8
QoS endpoint
Windows IBM Tivoli Monitoring for Transaction Performance 5.2:
QoS Endpoint
STI endpoints Windows IBM Tivoli Monitoring for Transaction Performance 5.2:
STI endpoint
Windows 2000 Resource Kit Windows Support Tools MKS Toolkit V8
Microsoft SQL Server 2000 IBM TBSM Base Services BSM3
TBSM console and propagation server
Windows 2000 Advanced Server
Windows 2000 Resource Kit Windows Support Tools MKS Toolkit V8
IBM TBSM Base Services BSM5
TDW database server
TDW Control Server
Windows 2000 Server DB2 Universal Database Server V7.1 FixPack 5 IBM Console Presentation Services
Tivoli Data Warehouse enablement packs
4.4 IBM Tivoli Monitoring profiles
IBM Tivoli Monitoring profiles are Tivoli Management Framework based profiles.
These profiles are created using the Tivoli Desktop interface. Each profile is created inside a container called a Profile Manager. The Profile Manager lists the potential subscribers for the profile. The IBM Tivoli Monitoring profile object is called Tmw2kProfile.
4.4.1 Profile Managers and IBM Tivoli Monitoring profiles
A Profile Manager window is shown in Figure 4-9 on page 102.
BSM6
TSLA database server
TSLA admin server TSLA reporting Server
Windows 2000 Server DB2 Universal Database Server V7.1 FixPack 5 IBM Tivoli Service Level Advisor 1.2 Server IBM Console Presentation Services
IBM Tivoli Service Level Advisor 1.2 Task Drivers IBM Tivoli Service Level Advisor 1.2 Reports
BSM, BSM7 Web and Web Application Servers
Windows 2000 Server DB2 Universal Database Server V7.1 FixPack 5 IBM HTTP Server V1.3.16
WebSphere Application Server V4.0.3 Tivoli Management Agent
IBMTIV2 Database Server
AIX 4.3.3 DB2 Universal Database Server V7.1 FixPack 5 Tivoli Management Agent
Machine name Operating System Software list
Figure 4-9 Profile Manager
We created a Tmw2kProfile for each type of monitoring that we use. The profile consists of the list of resource models contained within it. For each resource model, the following information is specified:
The indications or metrics that are evaluated in the resource models
The threshold of each indication and how many violations cause an alert (see Figure 4-12 on page 105)
The responses that can be triggered
The logging option for the resource model
The window to define the Tmw2kProfile object is shown in Figure 4-10 on page 103.
Figure 4-10 Creation of Tmw2kProfile
Click Add. We will then add a new resource model to the Tmw2kProfile object.
The Add resource model window is shown in Figure 4-11 on page 104.
Figure 4-11 Resource model configuration window
These are the descriptions of the fields and buttons for the Add Resource Model window shown in Figure 4-11:
Category Lists the existing resource model categories depending on the installed products.
Resource Model Lists the resource models in the selected category.
Cycle time Determines the interval at which the resource model will check its indications.
Threshold list Lists possible indications and their thresholds; when selected, the threshold entry is shown.
Indication button Opens the indication setting dialog.
Parameters button Sets monitoring filters and logging parameters.
Schedule button Determines the time interval within which this profile can be active.
Logging button Offers a selection of logging options to use.
The Indication window is shown in Figure 4-12..
Figure 4-12 Indication setting
In Figure 4-12, the following can be configured:
Occurences and holes setting
Event triggering setup
Action tasks
In Figure 4-13 on page 106, the parameter dialog is shown.
Figure 4-13 Parameter setting
The parameter setting window in Figure 4-13 allows you to select which metrics will actually be logged in the historical database and also the monitoring filter, in this case the Application Names to monitor for SQL statement activity.
The logging option is shown in Figure 4-14 on page 107.
Figure 4-14 Logging option
You should specify the logging option to log TEDW data for collection to Tivoli Data Warehouse.
After you have all the resource models defined, back in the Tmw2kProfile window (as shown in Figure 4-10 on page 103), click Edit -> Properties and enable the event forwarding function, as shown in Figure 4-15 on page 108.
Figure 4-15 Event setting
You also need to enable the forwarding setting. Either forward to IBM Tivoli Enterprise Console or to IBM Tivoli Business Systems Manager directly using Common Listener. We decided to use the interface to IBM Tivoli Enterprise Console.
4.4.2 Detailed profile setting
In this section, we show the detailed settings of the Resource Models in our Tmw2kProfiles objects. The settings are selected based on the monitoring need.
All these events are sent to IBM Tivoli Enterprise Console.
Apache Web Server monitoring
Apache Web Server monitoring provides the following functions:
Makes sure that the process is always running; the operator needs to be notified when the process is not running and cannot be started automatically.
Counts the hit rate for reporting, then reports any exceptionally high hit rate.
Reports excessive numbers of error requests, perhaps resulting from a Denial of Service attack.
We create the profile ITM_Apache_Pr with the content shown in Table 4-4.
Table 4-4 Monitoring resource models
WebSphere Application Server monitoring
WebSphere Application Server is a critical piece of the application. All the transactions are handled there. The monitoring aim is to maximize the availability and performance of the Web transaction. Therefore, the following monitoring is performed:
Monitor and log the status of the WebSphere Application Server
Monitor and log the JVM memory usage to ensure that there is enough storage available for the application
Monitoring for DB2 connection contention
Monitoring for number of HTTP sessions
The ITM_WAS_Pr profile that we define to address the WebSphere component is summarized in Table 4-5 on page 110.
Note: Depending on the setup of your environment, you may want to have the authentication fail if it is performed by the Web server; a typical
implementation performs authentication outside of the Web server itself.
Resouce model Indication and threshold Logging attribute Apache_
Note: Our application runs as servlets, without EJB; therefore, we only use the WebApplication monitor, without the EJB monitor or transaction monitor.
Table 4-5 Monitoring resource models
DB2 monitoring
The database monitoring needs to ensure the performance of the database to fulfill requests from the WebSphere Application Server. This result in the following monitoring:
Monitoring and logging of the availability of the DB2 instance
Monitoring of the Buffer Pool hit ratio, one of the most important parameters of DB2 performance
Monitoring of the DB2 subsystem connection and tablespaces status
Monitoring of locks and deadlocks
The DB2 monitoring details are shown in Table 4-6 on page 111.
Resouce model Indication and threshold Logging attributes WebSphereAS_
Table 4-6 Monitoring resource models
Windows server monitoring
For Windows server, we monitor the following attributes:
CPU utilization
Memory usage
Free disk space
Network card activity
Resouce model Indication and Threshold Parameters DB2 Buffer Pools High_AvgPoolReadTime 200
High_AvgPoolWriteTime 200
DB2 Activity Old_LastBackupTimestamp 2 High_PctConnectionsUsed 80
Table 4-7 Monitoring resource models
UNIX server monitoring
For the UNIX server, we monitor the following attributes:
CPU utilization
Memory usage
Free disk space
Network card activity
The contents of the ITM_UNIXOS_Pr profile are shown in Table 4-8.
Table 4-8 Monitoring resource models
4.5 IBM Tivoli NetView monitoring
To enable access to NetView from a remote Web console (either with the Web Console Application or from within a browser applet), be sure to register and start the netviewd daemon. You can do this by issuing \usr\OV\bin\ovaddobj
\usr\OV\lrf\netviewd.lrf from a command line.
Next, define at least one user by using the nvsetup application in order to gain access to NetView through the Web console. You will need a user definition to access NetView data from within IBM Tivoli Business Systems Manager.
To add a user, execute the \usr\OV\bin\nvsetup utility or click Tivoli NetView ->
Resouce model Indication and Threshold Parameters
Memory PctMemoryFree 10
-Network Interface Card
Utilization 90%
Physical Disk Free space 50 MB C:\
Processor CPU utilization 95%
Resouce model Indication and Threshold Parameters
CPU Utilization 95%
File System Free space 50 MB /var, /home/db2inst1
Memory PctMemoryFree 10
-Network Interface Utilization 90%
-Figure 4-16. In this dialog, select Web Server from the drop-down list and click Configure Web Console Security.
Figure 4-16 Invoking Web console Security from nvsetup
Alternatively, you can start the configuration utility from the native NetView console by clicking Administer -> Security Administration -> Web Console Security. In both cases, the Java-based Web Console Security application opens with the initial dialog box shown in Figure 4-17 on page 114.
Figure 4-17 The Web Console Security dialog box
In this dialog, you can add and edit users, roles, and views. Clicking Selected ->
Add or selecting the Add icon in the button row will open the Add User dialog shown in Figure 4-18.
Figure 4-18 The Add User dialog box
Enter a user name and a password. For now, assign the SuperUser role to the new user. Because an IBM Tivoli Business Systems Manager user might need to access all of the network topology, select No scoping restrictions for the Scope field. Click OK to close the dialog box, then click File -> Save to save the new user definition. You will be asked to restart the Web server to activate the changes. Reply Yes to restart the Web server.
Now from the NetView setup window, go to the Discovery tab and click Edit for the seed file. The seed file editor is shown in in Figure 4-19.
Figure 4-19 NetView seed file editor
We want to limit the discovery to our environment only, as shown in Figure 4-20 on page 116.
Figure 4-20 Adding discovery filter to our subnet only
Configuring IBM Tivoli Enterprise Console forwarding is done by clicking Tivoli NetView -> Administration -> Configure IBM Tivoli Enterprise Console Adapter. The following configuration assumes that we have a Managed Node installed in the ibmtiv5 where we run NetView. Without this, you would need to use a connection that is not secure by specifying the IP address of the IBM Tivoli Enterprise Console, its operating system, and its port.
The configuration of the adapter is done through the dialog shown in Figure 4-21 on page 117.
Figure 4-21 Configuring TEC adapter
As shown in Figure 4-21, the only events that we will send are the Link_Down and Link_Up events and those will be sent for all the SmartSets.
Because commands launched from IBM Tivoli Business Systems Manager to NetView are executed locally, the NetView Web console must be installed on all workstations that will be used to access IBM Tivoli Business Systems Manager and its Java console. NetView provides a download page with install images of the NetView Web console for various platforms. You can access this download page by connecting to NetView Server using a Web browser. In our case, the download page is:
http://ibmtiv5:8080/download
and 8080 is the default port on which the NetView Web server listens. A download page is shown in Figure 4-22 on page 118.
Figure 4-22 The NetView Web Console download page
For a Windows-based workstation, select nvwcinstal.exe and download the file.
This is a Windows install image which you can install on the target workstation.
Note: With the current implementation of the NetView IBM Tivoli Business Systems Manager adapter, a launch of the NetView Web Console from the IBM Tivoli Business Systems Manager Java console will not work with the default path the Web Console installer suggests. One component installed on the IBM Tivoli Business Systems Manager side, nvlaunch.jar, only supports file paths in the classic 8.3 notation. We recommend that you change the install directory during the NetView Web Console install to something like c:\NVWC, as shown in Figure 4-23.
Figure 4-23 Changing the default path for the NetView Web Console
The install procedure creates an object on your Windows desktop. Double-click that icon to launch the Web Console and open the login window as shown in Figure 4-24.
ibmtiv5
Type in your user ID, password, the hstname of your NetView sever, and the port on which the NetView Web Console is listening.
After a successful logon, select File -> Open from the menu to display a map inside the Web Console as shown in Figure 4-25.
Figure 4-25 Open a map
4.6 Web transaction response time monitoring
We need to define and manage the response time for the Web Services transactions. We capture this information using the IBM Tivoli Monitoring for Transaction Performance: Web Performance. This product used to be called the Tivoli Web Services Manager.
There are two tools that we used:
The Quality of Service monitor, which collects a sample response time of actual end-user experiences for browsing the Web site
The Synthetic Transaction Investigator monitor, which monitors the availability and response time for a certain transaction
The IBM Tivoli Monitoring for Transaction Performance uses the Web browser to administer its function. We log in to the Internet Management Server as shown in Figure 4-26. The URL for us is http://bsm9/.
Figure 4-26 Log in to the Tivoli Internet Management Server
The initial menu that we were presented with is shown in Figure 4-27 on page 122.
Figure 4-27 Initial TMTP menu
4.6.1 Quality of Service monitoring
We create Quality of Service jobs to monitor the critical transaction response time on our Web servers. The jobs are called:
QOS_bsm2_transfer_01QOS job for bsm2 endpoint with transaction transfer
QOS_bsm7_transfer_01QOS job for bsm7 endpoint with transaction transfer
QOS_bsm2_eopen_01QOS job for bsm2 endpoint with transaction eopen
QOS_bsm7_eopen_01QOS job for bsm7 endpoint with transaction eopen The following steps are taken to create the QOS jobs:
1. From the menu, select Quality of Service -> Create Quality of Service Job.
2. Assign the endpoint for the job as shown in Figure 4-28 on page 123. Click Next.
Figure 4-28 QOS job - choose endpoint
3. Assign the Schedule for the job as shown in Figure 4-29; the definition shown is for a continuous job. Click Next.
Figure 4-29 QOS job - define schedule
4. Assign the parameters for the job; we select the target URI condition for the transaction that we want to monitor, as shown in Figure 4-30 on page 124.
Click Next.
Figure 4-30 QOS job - job parameter
5. Define the monitoring constraint (threshold) and event severity to be generated, as shown in Figure 4-31 on page 125. Click Next.
Figure 4-31 QOS job - define constraint
6. Save the job as shown in Figure 4-32. Click Save.
Figure 4-32 QOS job - saving
7. Click Quality of Service -> Configure Events to configure event forwarding
Figure 4-33 QOS events configuration
4.6.2 Synthetic Transaction Investigator monitoring
We also create Synthetic Transaction Recording for both the transfer and eopen transactions using the STI Recorder, as follows:
1. Download the STI Recorder from the Web interface by clikcing Synthetic Transaction Investigator -> Download Transaction Recorder.
2. Start the Recorder by selecting Programs -> Tivoli -> Synthetic
Transaction Investigator Recorder. The login dialog is shown in Figure 4-34 on page 127. It is the Web target realm login, not the Management Server login.
Figure 4-34 Login to STI Recorder
3. Record the sample transaction flow and save it to the Management Server by clicking the Save Transaction button. Authentication to the Management Server is now required. The Save Transaction dialog is shown in Figure 4-35 on page 128.
Figure 4-35 Save a transaction
The STI jobs that we create are called:
STI_bsm2_transfer_01STI job for transfer to bsm2 endpoint
STI_bsm7_transfer_01STI job for transfer to bsm7 endpoint
STI_bsm2_eopen_01STI job for eopen to bsm2 endpoint
STI_bsm7_eopen_01STI job for eopen to bsm7 endpoint The following procedure creates the STI jobs:
1. Click Synthetic Transaction Investigator -> Create Transaction Playback.
2. Specify the endpoint to use as shown in Figure 4-36 on page 129. Click Next.
Figure 4-36 STI job - specify endpoint
3. Specify the schedule to use as shown in Figure 4-37. Click Next.
Figure 4-37 STI job - continuous schedule
4. Specify the transaction to play back as shown in Figure 4-38 on page 130.
Click Next.
Figure 4-38 STI job - recorded transaction
5. Specify the proxy information as shown in Figure 4-39. Click Next.
Figure 4-39 STI job - proxy
6. Specify the constraint as shown in Figure 4-40 on page 131. Click Next. Here you can define the response time measurement, HTTP return code and
Figure 4-40 STI job - constraint definition
7. Specify the job name to use as shown in Figure 4-41. Click Save.
Figure 4-41 STI job - saving
8. Configure STI events by clicking Synthetic Transaction Investigator ->
Configure Events. The events that we choose for forwarding are shown in Figure 4-42.
Figure 4-42 STI events configuration
4.7 Defining TEC rules
Since we decided to use IBM Tivoli Enterprise Console to transfer all events from the monitoring systems to IBM Tivoli Business Systems Manager, we need to prepare the IBM Tivoli Enterprise Console.
The IBM Tivoli Enterprise Console is based on a set of definitions called a rule base. A rule base contains event format definitions in a format called baroc and processing logic written in the Prolog language called rulesets. This information needs to be customized to be able to send events to IBM Tivoli Business Systems Manager.
4.7.1 Adding IBM Tivoli Monitoring rules
For the IBM Tivoli Monitoring product modules, there are default tasks, baroc files and rulesets that are supplied by IBM. These supplied baroc files and rulesets are listed in Table 4-9 on page 133.
Table 4-9 Files to be imported to rule base
The tasks or scripts provided in Table 4-9 allow the creation of new rule bases with the necessary baroc and ruleset files.
4.7.2 IBM Tivoli Monitoring for Transaction Performance rules
For IBM Tivoli Monitoring for Transaction Performance, a baroc file is supplied in the Internet Management Server, at $INSTHOME/TIMS/lib/TranPerf.baroc. You need to include this baroc file in the final rule base manually. There are no default processing rulesets for these events.
From 4.6, “Web transaction response time monitoring” on page 120, we forward the following events to the IBM Tivoli Enterprise Console (both the violation and recovery events):
QoS Backend Service Time
QoS Page Display Time
QoS Round Trip Time
STI Round Trip Time
STI Overall Transaction Time
STI Undesired Content found
STI URL not available
IBM Tivoli Monitoring for Baroc and ruleset files location Task or script IBM Tivoli Monitoring for
IBM Tivoli Monitoring for Baroc and ruleset files location Task or script IBM Tivoli Monitoring for