Archive log files store database transactions history and can be used for database recovery operations. They are important files in any database environment. These files can be placed to slower but larger disks as I/O performance is not a major factor for a disk volume where archive log files are stored.
Tivoli Netcool Performance Manager database during normal operation generates tens of GBs of Archive log files per day. As a result, a large disk volume is required to store them.
Hardware
v Slower but larger SATA disks (7200 RPM) of appropriate total size must be grouped into one RAID-10 array.
Configuration
v 1 LUN is exported from the disk array that is created in the previous step and mounted to /oradump/archivelogs mount point.
Database backup files
Database backups are an important part of any production environment and must be taken regularly.
One backup approach for backing up the Tivoli Netcool Performance Manager database environment is a proxy full database backup that is stored on disk, with possible subsequent backups to tape to increase data safety. The backup database image is updated on disk regularly by applying recent transactions from the archive log files.
To store a backup copy, enough disks must be allocated to store files of approximately the same size as the total Tivoli Netcool Performance Manager database plus 10%.
See Chapter 6, “Backup strategy,” on page 27, for details on backing up the database.
Hardware
v Slower but larger SATA disks (7200 RPM) of appropriate total size must be grouped into one RAID-10 array.
Configuration
v 1 LUN is exported from the disk array that is created in the previous step that is mounted to /oradump/backups mount point.
Log files
Tivoli Netcool Performance Manager maintains all its application log files in /appl/virtuo/logs. A cron job, created during installation, archives all log files to /data/trace_archive1by default (in gzip format). You must manage the available file system space where the /data/trace_archive1 directory is stored .
Hardware
In most cases, both directories can be stored on the root file system. If
/data/trace_archive1needs to be preserved for a long time, slower but larger SATA disks (7200 RPM) of appropriate total size must be grouped into one RAID-1 or RAID-10 array, either on the SAN or internal to the application server computer.
Configuration
In most cases, both directories can be stored on the root file system.
Alternatively, 1 LUN is exported from the disk array that is created in the previous step and mounted to /data/trace_archive1 mount point.
Report results
Tivoli Netcool Performance Manager report results are stored in /appl/virtuo/var/rg.
This directory contains two sub directories:
v Logs - report result logs.
v Spool - result results (XML files) and exports (CSV, XML, XLS).
These directories are not archived by the system. You must manage the available file system space where these directories are stored.
Configuration
In most cases, both directories can be stored on the /appl file system.
Alternatively, 1 LUN is exported from the disk array that is created in the previous step and mounted to /appl/virtuo/var/rg mount point.
Hardware
In most cases, both directories can be stored on the /appl file system. If you want to preserve reports and their associated logs for a long time, slower, group larger SATA disks (7200 RPM) of appropriate total size into one RAID-1 or RAID-10 array. You can store them either on the SAN or internal to the application server computer.
Chapter 4. File types and disk layout 15
Loader files (LIF files)
Loader files, or LIF files, are generated by the Gateway component and contain formatted raw performance data from the network. These files are processed by the data loader processes and the information is stored in the database.
The loader LIF files are stored in /appl/virtuo/var/loader/spool by default. As LIF files can take a large amount of disk space (especially in a backlog scenario), it is important to carefully select where they are located. Depending on the
deployment topology (monolithic or distributed) and the available hardware, LIFs can either be on the Loader servers or on the SAN.
The number of days LIF files are kept is configurable. The number of days to keep LIF files is partly determined by the frequency of backups. If backups are taken weekly, you must keep eight days of LIF files so that the LIF files can be reloaded easily. Normally three days are acceptable but it varies from deployment to deployment.
LIF file size must be carefully assessed. When you move from legacy products, you must know LIF file sizes.
You can estimate the daily LIF file size from the EDDS (Estimated Daily Database Size). 10 GB of EDDS normally corresponds to 70 GB of LIF files (it can vary depending on the Technology Packs and the percentage of counters loaded).
For example, for a system with an EDDS of 100 GB and a 5-day backlog retention policy, the LIF file size might be estimated to 100*7*5 = 3.5 TB logical, and 7 TB when taking into account RAID-1. 100 GB of EDDS requires a minimum of eight disks (30 GB EDDS can saturate two disks in RAID-1), a possible configuration includes eight 1-TB SATA-2 disks, or preferably 16 500-GB disks. The disks are configured in RAID-10 to take into account extra IO during backlog processing.
Backlog scenarios need to be taken into account when you compute the total disk space required for storing LIFs.
When you store LIF files, you must also consider the large number of concurrent random reads and writes. 30 GB daily data size (EDDS) can saturate the IO capabilities of two 7200-RPM SATA-2 disks that are configured in RAID-1.
Configuration
Depending on the configuration, the disks that hold the LIFs can be stored on the Loader servers.
Alternatively, 1 LUN is exported from the disk array that is created in the previous step and mounted to /appl/virtuo/var/loader/spool.
Alternatively, 1 LUN is created for each Technology Pack or data source and mounted to /appl/virtuo/var/loader/spool/<techpack or datasource>.
Hardware
Several large SATA-2 disks that are configured in RAID-1 or RAID-10 (depending on size and configuration).
Gateway files (raw files)
Raw files are generated on the gateways, converted to LIF files and transferred to the application server. They must be maintained on the gateway until the day's data is successfully loaded into the database. Raw, gateway files must be placed on RAID-1 devices.
The gateways are installed in $WMCROOT/gways. All gateway framework files are in the directory: $GATEWAY_ROOT.
Each gateway has spool directories that are configured for input files, intermediate files, and loader files.
v IN_DIR=/spool/input_d v INT_DIR=/spool/inter_d v OUT_DIR=/spool/output_d
Spool directories must be separated from the main installation directory to avoid memory issues as spool directories grow. The preceding example assumes that spool directory is created from root directory.
Disk layout on the database server
A list of the minimum disk requirements for the database component is provided in the following table. The minimum files system sizes that are provided are only a minimum and the file systems are sized according to the customers-specific
requirements.
Details on the RAID type and LUN setup that is required for each file system and data file type are given in the previous sections.
Table 5. Minimum disk requirements
File system Software Typical Size
Minimum Disks
/oralogs1 Oracle redo log location 1
8 GB 2 1
Chapter 4. File types and disk layout 17
Table 5. Minimum disk requirements (continued) File system Software Typical Size
Minimum Disks
Required RAID Type /oralogs2 Oracle redo log
location 2
32 GB (min) Total size is calculated based
32 GB (min) Total size is calculated based
32 GB (min) Total size is calculated based
Chapter 5. System configuration
File system, network, and database configurations.
File system and SAN configuration
The following are suggested file system and SAN configurations:
v On Linux, all database server file systems must be configured with file system type ext3. For performance reasons, access time logging for files and directories must be removed (defaults, noatime, nodiratime).
v ext3 supports three types of journaling modes:
v Journal - has higher performance overhead.
v Ordered - is the default setting and suggested option.
v Writeback - provides the fastest access to the data at the expense of data consistency.
v On Solaris, all database file systems (/oradata*, /oralogs*, and /oradump) must be configured as UFS, while all other application-related file systems (/, /appl, and /opt) must be configured as ZFS.
v A minimum of 2 GB of cache is required per Fibre Channel controller.
v Direct-IO must be enabled for Oracle by setting the Oracle filesystemio_optionsparameter:
filesystemio_options = setall
v Individual SAN LUNs must be created for each /oradata01.04 File system, with a minimum Stripe/Segment Size of 512 KB and RAID type 1+0 (mirroring and striping).
v Oracle redo logs (oralogs1 and oralogs2) must be placed on separate LUNs.
Network configuration
The following are suggested network configurations:
v In gigabit networks, large maximum transmission units (MTU) sizes (also known as JumboFrames) can provide better network performance. To transfer large amounts of data at gigabit speeds, increasing the default MTU size can provide significant performance gains. A sample MTU Jumbo size is 9000.
v The Underlying IP Network must support 1 GB/S transfer rates.
Database configuration
Database memory, database redo logs and database temporary tablespace configurations.
© Copyright IBM Corp. 2007, 2015 19
Database memory configuration
Following are the main memory components for Oracle:
v SGA (Shared Global Area) v PGA (Program Global Area)
Both parameters depend on the amount of data loaded each day into the database and are calculated as part of the Dimensioning procedure.
8 GB of the target server must be reserved for the OS and background tasks.
SGA
The SGA is calculated as part of the Dimensioning procedure. The Dimensioning procedure estimates the amount of daily loaded data (EDDS - Estimated Daily Database Size) based on network size. The SGA is then calculated at 1.3 * EDDS.
The SGA is made up of two main components, SHARED_POOL_SIZE and DB_CACHE_SIZE.
v DB_CACHE_SIZE- for optimal performance, store a full day of data in to the database cache. Because of the load patterns, an undersized database cache might double most IOs, and a READ bottleneck on the /oradata* file systems during loading, especially later in the day.
For the resizing of an existing system, the required database buffer cache can be computed from the size of the daily partitions of a running system.
v SHARED_POOL_SIZE- for Tivoli Netcool Performance Manager, a minimum of 3 GB must be configured for the SHARED_POOL_SIZE.
Automatic shared memory management:
Automatic Shared Memory Management (ASMM) is Oracle automatic management of SGA memory.
Important: ASMM must not be used with Tivoli Netcool Performance Manager because of Tivoli Netcool Performance Manager use of generated SQL and because memory would not be used optimally on the system workloads.
The SGA value that is an output from the Dimensioning procedure is the total size of SGA. ASMM is configured ON by default in Tivoli Netcool Performance Manager and must be turned OFF after installation. The total size of the SGA must be divided into specific SGA parameters that are buffer cache and shared pool size.
The following table provides an example of SGA parameter settings that are used for a server. The server is dedicated to the database, with 128 GB RAM, 80 GB SGA, and 40 GB PGA (8 GB of the total RAM is reserved for the OS).
Table 6. Breakdown of database SGA parameters
Parameter Setting Details
sga_max_size 81920M Set by Oracle to total SGA
size.
sga_target 0 Disables ASMM when set to
0.
shared_pool_size 3G Set to at least 3 GB.
Table 6. Breakdown of database SGA parameters (continued)
Parameter Setting Details
db_cache_size 77G Available buffer pool cache
memory.
log_buffer 256M Dependent on system
resources and load activity.
Normally 11 - 400 MB.
java_pool_size 100M Set to 100M for Java pool.
large_pool_size 100M Set to 100M for large pool.
To disable ASMM, run the following commands that are connected to the database with SYSDBA privileges:
ALTER SYSTEM SET sga_target=0 SCOPE=spfile sid='*';
ALTER SYSTEM RESET sga_max_size SCOPE=spfile sid='*';
ALTER SYSTEM SET db_cache_size=<db_cache_size> SCOPE=spfile sid='*';
ALTER SYSTEM SET shared_pool_size=<shared_pool_size_at_least3G> SCOPE=spfile sid='*';
ALTER SYSTEM SET large_pool_size=100M SCOPE=spfile sid='*';
ALTER SYSTEM SET java_pool_size=100M SCOPE=spfile sid='*';
The database must be restarted to enable these changes.
PGA: PGA is used by Oracle for sorting data for individual sessions. Normally the PGA is set to approximately 20% of the SGA. Because of the number of large sorts on a typical Tivoli Netcool Performance Manager database, the PGA must be set to approximately 25% of the database cache.
The following table provides an example of PGA parameter settings that are used for a server that is dedicated to the database. The server has 128 GB RAM, 80 GB SGA, and 40 GB PGA (8 GB of the total RAM is reserved for the OS).
Table 7. Breakdown of PGA parameters
Parameter Setting Details
pga_aggregate_target 40G Target aggregate PGA
memory available to all server processes.
Database redo logs
Oracle redo logs record all changes to the data files. The normal redo log setup consists of five groups; each group has two members. These members are split between the two locations, /oralogs1 and /oralogs2.
Five redo log groups are normal for systems that are going to be set up in archive log mode. It is suggested when the system is being backed up.
The size of the redo logs depends on the amount of activity on the system. It is usual for redo log groups that switch once every 20 minutes. The size of the system affects the switching frequency.
For small to medium systems, the redo logs can be set between 400 MB and 1G for each member. It results in 400 MB * 2 members * 5 groups => 4 GB. 4 GB is divided evenly between /oralogs1 and /oralogs2.
Chapter 5. System configuration 21
For a larger system, the redo logs can be sized at 3 GB each for each member. The results in 3 GB * 2 members * 5 groups => 30 GB. 30 GB is divided evenly between the /oralogs locations.
The configuration and sizing of the database redo logs are updated and delivered to the customer in a customer database template, vtdb_<customer_name>.dbt.
See section “Updating customer database template” on page 34, for information on updating the database redo logs with the correct sizings and locations.
Database temporary tablespace
Oracle temporary tablespaces are used to manage space for database sort
operations and for storing global temporary tables. If a large sort operation cannot fit in memory, then space will be allocated in a temporary tablespace for doing the sort operation.
In Tivoli Netcool Performance Manager , large reports or complex summaries, which join across a number of large tables will require a large temporary tablespace. A suitably sized TEMP tablespace is 80 GB-100 GB.
Application server configuration
On the Application Server component, JBoss must be configured with 3 GB of Java heap space (“–Xms3G –Xmx3G”). This setting can be configured in the file:
$WMCROOT/conf/as/jvm-default.ini.
To change the setting to 3G, make the following update to the jvm-default.ini file:
-Xms3G -Xmx3G -XX:+AggressiveHeap -XX:MaxPermSize=96m
The Security Directory Service, which is required for LDAP authentication and authorization, uses approximately 2 GB of Random Access Memory (RAM), other Tivoli Netcool Performance Manager Java processes use approximately 2 GB.
An extra 8 GB RAM must be reserved for the OS and background tasks.
Loader configuration
The number of loaders to configure on a system depends on the number of technology packs and the amount of data to load on the system.
There is a minimum of one loader per datasource, and in most cases two datasources per technology pack:
v The Performance Counter datasource v The Hierarchy datasource
Certain technology packs, such as the Ericsson UTRAN Technology Pack, have more datasources, the PDF counter datasource in the Ericsson UTRAN Technology Pack.
Most small datasources (such as the hierarchy datasource or the performance counter datasource for small technology packs with few entities), require only one single-threaded loader with 512 MB to 1 GB of memory. For example, JVM arguments: “–Xms1G –Xmx1G”.
Larger datasources require the use of one or more multithreaded loaders. For example, it is possible to configure one multithreaded loader with two LIF-parser threads and four DB-writer threads for each 10000 cell of an average technology pack. Each such loader requires more memory as they process several LIFs in parallel. For example, “–Xms3G –Xmx3G” to configure 3 GB of memory.
The following list gives an example of a possible configuration for a system with 20000 Nokia UTRAN cells and 10000 Ericsson cells with PDF counters. The information about memory requirements is provided from the Dimensioning procedure, and depend on the actual network size, the load interval (for example, 15 minutes), and the percentage of counters that are loaded.
Nokia hierarchy One single threaded loader with 512 MB RAM Nokia PM Two multithreaded loaders with 3 GB RAM each Ericsson hierarchy One single threaded loader with 512 MB RAM Ericsson PM One multithreaded loader with 3 GB RAM Ericsson PDF One multithreaded loader with 3 GB RAM
In this example, the system would consist of a total of 6 loaders and 13 GB of RAM.
The following must also be set:
v The ’insert.max.batch’ size from the lc_loader_config_properties table must be set to the default value of 5000.
v You must leave an extra 8 GB of RAM for the OS and background tasks.
v
– For loaders, allow 0.2 GB of memory for every 1 GB of data that goes into the database each day (EDDS). Minimum memory per loader (for example, for small hierarchy loaders) is 400 MB.
Gateway configuration
The Gateway framework uses the following configuration files. They are in the vstartdirectory:
v EngineConfig.pm– configuration file of the first stage of the Gateway.
v UserConfig.pm– a user configurable Perl module for configuring the Gateway Post Parser.
v TransferConfig.pm– configures the transfer in of raw files and transfer out of processed LIF files.
v Gateway_start.sh– this script is used to start the Gateway.
Within the Gateway configuration, there are configuration directories for every vendor subsystem and data revision. They are contained within a directory called config. Following are the contents of these directories:
v The docs directory contains documentation on the configuration for each vendor data revision supported.
v The configuration directories are named based on the vendor subsystem, for example ericsson-bss.
Within each vendor subsystem directory, there are directories for each data revision supported. For example, r12_ascii, r12_asn1. These directories contain the
configuration files that are to be referenced by the Gateway framework to parse the vendor data. For example, EngineConfig.pm, UserConfig.pm,
StatisticsConfig.pm, TransferConfig.pm, NotificationConfig.pm.
Chapter 5. System configuration 23
The StatisticsConfig.pm, TransferConfig.pm, and NotificationConfig.pm files can be obtained from the Gateway framework example directory. If the Statistics Configuration is configured, the file_statistics and block_statistics directory must be created manually by the user, and the path that is specified in the
The StatisticsConfig.pm, TransferConfig.pm, and NotificationConfig.pm files can be obtained from the Gateway framework example directory. If the Statistics Configuration is configured, the file_statistics and block_statistics directory must be created manually by the user, and the path that is specified in the