I 5 (AS/400) INSTALLATION
7: BANDWIDTH MANAGEMENT
Overview
This section describes the bandwidth management features of NFM. This includes two parts. This first section, bandwidth control, describes the mechanisms used by NFM to tune the amount of bandwidth NFM uses. The second section, bandwidth monitoring/reporting, describes the tools available with NFM to examine its bandwidth usage.
Bandwidth Control
Overview
Performing file transfers across a network can obviously be very bandwidth consuming. This problem is of course even more intensified when transferring files to or from a node group. Left unchecked, NFM will attempt to do all transfers in a running plan, simultaneously, and as fast as is possible, being almost always limited in speed, to whatever the network can handle. The source node or nodes, will attempt to write all data until the TCP/IP stack halts the flow (flow control) and waits for the network to catch up. Because of these factors, doing file transfers can have a severe impact on other network activity.NFM provides two primary mechanisms that enable the user to control the amount of data NFM attempts to transfer at any given moment. This consists of limiting the number of active nodes and data pacing or throttling.
Maximum active nodes
NFM allows the user to configure the maximum active nodes in two places. There is an overall “Maximum Active Nodes” field in the system settings that can be used to limit the total number of nodes in use at a given time within all of NFM. Once a particular node starts a file transfer, it will reserve one of these slots until it is finished and then free up the slot for another node to begin. This limit is system wide; no matter how many plans may be actively trying to do transfers. Any node that is halted, waiting for a slot, will be clearly marked as such while monitoring a plan. (See “System settings” in Chapter 4, “Configuration,” for setting this field.)
In addition to this, there is a “Maximum Active Nodes” field that is available on a per node group basis. This works in the same way but allows the user to give different node groups, a different number of slots from which to work with. This allows assigning some groups with more bandwidth than others. (See “Node groups” in Chapter 4, “Configuration,” for more information.)
This provides an efficient way to increase performance for other network activity without necessarily impacting the performance of NFM. Other applications are able to better compete for network resources, but if no other activity is occurring, NFM will still try to use as much bandwidth as is available. In addition, even if NFM is the only application using the network, setting the
maximum active nodes may actually make NFM more efficiently use the existing bandwidth. This would be true if too many simultaneous network connections adversely affected the network as a whole. So, put another way, setting this value might make NFM finish a set of file transfers faster, than if the value was not set at all.
Data Pacing
NFM implements a configurable pacing method that can be tuned on an individual node basis. When this is active, a source node that is sending a file, will only send a certain number of data blocks, before waiting for a reply from the receiving node (the data block size is also configurable). Although this creates a slight overhead, it can dramatically increase performance of non-NFM network activity, because the NFM sending nodes will not be able to flood outgoing TCP/IP queues with data, without letting other applications get their data written out in a timely fashion.Like the “maximum active nodes,” this mechanism is extremely efficient because it will only significantly slow down NFM transfers if other network activity is present. (For more information on implementing this option, read the descriptions for the “Transfer Block Size” and “Transmit Window Count” fields for the applicable node models, under “Models” in Chapter 4, “Configuration.”)
Bandwidth Monitoring
Overview
NFM provides a reporting mechanism that offers detailed statistics about its own bandwidth usage. This can be used to analyze network usage and aid in tuning the bandwidth control mechanisms. In addition, the data can offer the user-detailed accounting of network usage by selecting and reporting the data using multiple criteria. The user could for example, report exactly how much bandwidth was used by a given node over a certain time period, or perhaps, how much bandwidth was used in transferring a particular set of files or even an individual file.The reporting is available in two ways. A performance graph is available through the GUI that can view real time statistics as they are occurring, or view historical data. Also the statistics information can be exported in a batch fashion to simple text files, where it would be available for outside applications. Both of these allow extensive selection criteria to decide what to report (by time interval, node, fileset, individual file, etc.). The amount of statistics data that is maintained can be configured by the user with the ability to automatically purge or compress and archive data automatically at set times.
Extracts
Extracts are configurable records that define the parameters for a query of performance data (whether through graph or batch). The extracts define two major parts of any given query; selection and formatting. The Selection part provides the ability to filter the available performance data to isolate only the data that is desired. The Formatting portion is used to create custom reports (in batch), or to indicate which binary fields are desired to look at (in graph form).Extracts may be accessed from the Performance drop down menu of the
Management sub-menu. The following fields and folders are maintained on
this screen:
Extract Name This is a user selected name for an extract. Type in a new entry, or select an existing entry from the drop down list.
Selection Click on this folder to display or edit the desired criteria for selecting performance data. The available fields and their field type are:
/Byte Count numeric (bytes)
/File Name string
/Fileset Name string
/Plan Name string
/Plan Instance numeric
/Source Node CPU Time numeric (milliseconds)
/Source Node Disk Read Time numeric (milliseconds)
/Source Node Name string
/Source Node Network Write numeric (milliseconds)
/Target Node CPU Time numeric (milliseconds)
/Target Node Disk Write Time numeric (milliseconds)
/Target Node Name string
/Target Node Network Write numeric (milliseconds)
The radio buttons for each field allow one of the following choices:
N/A Not applicable.
Equals or Starts With String selections.
<, =, >, Not Equal Numeric selections.
When a query is performed, the available data records must pass each of the above field selection checks to be included in the output.
Formatting Click on this folder to display or edit the desired formatting for a query done with this extract. The available fields and their usage is as follows:
Output File Format
Determines how the fields are separated for batch output. This may be set to one of the following: Comma separated, Tab
separated, or, Fixed Length. The Fixed Length will
use the length parameter specified per line entry (as discussed below).
Show Zeros
This checkbox will determine whether or not to generate entries that contain zero binary data, or to only display non-zero information (the default) with regards to batch output (has no effect on graph). The following example describes this better:
Let’s assume a file of size 100 bytes was transferred to a specific target node at 1:30, 2:30 and then 3:30 p.m. Now let’s say a query was performed to look at the byte count of transfers from 12:00 (noon) to 6:00 p.m., on a one (1) hour interval for just this target node. If this box is not checked then only the records containing data would be displayed something like the following (this particular display would show from an extract that included the fields
timestamp followed by byte count, with a comma
separated output format, see below):
03/01/10 14:00:00,100 03/01/10 15:00:00,100 03/01/10 16:00:00,100
Checking the Show Zeros option would cause an entry to be printed for each time interval regardless of the lack of actual performance data present as follows:
03/01/10 12:00:00,0 03/01/10 13:00:00,0 03/01/10 14:00:00,100 03/01/10 15:00:00,100 03/01/10 16:00:00,100 03/01/10 17:00:00,0 03/01/10 18:00:00,0
This option might be necessary for input to another application that expects to see uniform counts whether they are zero or not.
Fields
Multiple line entries may be entered in this portion of the folder to define the different fields (and their order) that will be output during a batch query. The binary fields chosen here will reflect which entries will show on a graph query (string entries cannot be graphed of course).
To add a new line (and thus a new field entry), fill in the appropriate fields in the bottom of the screen and click on the Add button. To change an existing line, click on the line to highlight it and change the appropriate fields at the bottom of the screen and then click on
the Change button. To change the order of the entries, highlight a
line and then click on Move Up or Move Down button. Use
Delete Line to delete a highlighted entry, and use the Clear
button, to remove all entries.
The available field entries that may be defined (as reflected in the
Field Name drop down box) are:
Timestamp Constant Active Nodes Byte Count File Name
Fileset Name Plan Name Plan Instance
Source Node CPU Time
Source Node Disk Read Time Source Node Name
Source Node Network Write Target Node CPU Time
Target Node Disk Write Time Target Node Name
Target Node Network Write Time
Most of these entries match the list of selection criteria described in the previous section.
Format
This field describes the display format used for this field in a batch query. The following choices are available:
ASCII Decimal, zero filled ASCII Decimal, space filled Constant
Active Nodes
Buttons
The following buttons are available on the Extracts screen:Apply Applies all changes made on this screen. This will cause an add or update to occur, depending on whether or not the extract preexisted.
Cancel Cancels all changes on this screen since the last time this extract was displayed or applied.
Delete Deletes the currently displayed extract.
Graph Takes the session directory to the Graph screen, where queries may be performed (described in the next section).