SCSI bus organization and speed

Concern often exists over the performance effects caused by the number of drives on the SCSI bus, or the speed at which the SCSI bus runs. Yet, in almost all modern server configurations, the SCSI bus is rarely the

If you wish to monitor disk activity you need to enable the physical disk counters. In Windows NT 4.0, physical disk counters are disabled by default. To enable them, issue the commandDISKPERF -Ythen restart the computer. In Windows 2000, physical disk counters are enabled by default.

Keeping this setting on all the time draws about 2-3% CPU but if your CPU is not a bottleneck, this is irrelevant and can be ignored.

TypeDISKPERF /?for more help on the DISKPERF command.

Activating disk performance counters

bottleneck. In most cases, optimal performance can be obtained by simply configuring 10 drives per Ultra SCSI bus. If the application is

byte-I/O-intensive (as is the case with video or audio) five drives on one Ultra2 SCSI bus can be used for a moderate (10-20%) increase in system performance.

In general, it is rare that SCSI bus configuration or increasing SCSI bus speed can significantly improve overall server system performance.

Consider that servers must access data stored on disk for each of the attached users. Each user is requesting access to different data stored in a unique location on the disk drives. Disk accesses are almost always random because the server must multiplex access to disk data for each user. This means that most server disk accesses require a seek and rotational latency before data is transferred across the SCSI bus.

7.7.6.1 SCSI bus speed

As described in 7.7.3, “Drive performance” on page 143, total disk seek and latency times average about 8-12 ms, depending on the speed of the drive.

Transferring a 2 KB block over a 40 MBps SCSI bus takes about 0.5 ms, or approximately 1/20th of the total disk access time. It is easy to see that increasing the bus speed to 80 MBps will only improve the 1/20 portion of time to roughly 1/40 of the total time, resulting in a small fractional gain in overall performance.

Tests have shown that for random I/O, drive throughput usually does not approach the limits of the SCSI bus.

In some cases, Ultra2 SCSI (80 MBps) can be shown to offer measurable performance improvements over Ultra SCSI (40 MBps). This usually occurs when measurements are made with a few drives (four to eight) running server benchmarks that transfer large blocks of data to and from the disk drives.

System performance gains can be in the range of 5-10%. Typical examples are file-serving and e-mail benchmarks, where data transfer time becomes a larger component of the total disk access time.

File-serving and e-mail benchmarks (and applications) transfer relatively large blocks of data (12 KB to 64 KB), which increases SCSI bus utilization.

More importantly, however, these benchmarks usually build a relatively small set of data files, resulting in artificially reduced disk seek times. In production environments, disk drives are usually filled to at least 30-50% of their

capacity, causing longer seek times compared to the benchmark files that might only use 2-3% of the disk capacity. After all, building a 2 GB database

for a benchmark might seem like a large data set, but on a disk array

containing five 9 GB drives, that database utilizes less than 1/20th of the total space. This greatly reduces seek times, thereby inflating the performance contribution of the SCSI bus.

Most IBM SCSI drive enclosures offer the ability to split the backplane to offer dual SCSI bus capability. This effectively provides the same performance of Ultra2 by using dual Ultra SCSI buses in one drive package. In the case of the EXP200, which supports Ultra2, the backplane can be split to provide throughput benefits similar to Ultra3 SCSI, provided that the other components in the system can handle this level of throughput.

7.7.6.2 PCI bus

Don't forget that all this data must travel though the PCI bus. The peak data transfer rate for 32-bit 33 MHz PCI is 132 MBps, but the maximum sustained rate is only 80-100 MBps. The ServeRAID-2 adapter used three 40 MBps Ultra SCSI buses, which had the potential to sustain peak rates of 120 MBps.

This transfer rate was perfectly matched with 32-bit 33 MHz PCI. Using Ultra2 SCSI only provides the possibility of improving maximum sustainable

performance when both the adapter and the system support faster data transfer rates.

The ServeRAID-3HB adapter offers Ultra2 SCSI transfer performance and can transfer data over 64-bit PCI, which is better matched to the transfer requirements of its three Ultra2 SCSI buses. The adapter must be plugged into a 64-bit slot for Ultra2 SCSI capabilities to take maximum advantage of its potential.

Ultra3 SCSI's 160 MBps rate has similar issues: even faster PCI-to-memory performance is required before maximum throughput can be achieved. A three-channel Ultra3 SCSI RAID adapter can potentially deliver peak rates of 480 MBps. For the moment, no PCI interface can offer such throughput performance. In fact, most memory subsystems cannot offer that much bandwidth for all of the PCI slots combined.

All this is not to say that Ultra2 and Ultra3 SCSI performance have no place on servers. Several issues must be addressed. You should simply remember that these are important considerations when configuring a balanced server.

Spec-driven technologies, such as SCSI, are often motivated by desktop environments. In the desktop environment, applications tend to be more sequential, and the system usually has a single SCSI adapter that can monopolize much of the PCI to memory bandwidth. In the desktop

environment, SCSI-3 provides significant performance gains. Because of the more random nature of server applications, these benefits often do not translate to server environments. The entire delivery path from memory through the PCI bus, over the adapter, and out to the drive must be optimized before faster SCSI bus speeds will realize any appreciable system

performance gains for any workload.

7.7.6.3 Multiple SCSI buses

The SCSI bus organization of drives on a multi-bus controller (such as ServeRAID) does not significantly affect performance for most server workloads.

For example, in a four-drive configuration, it doesn’t matter whether you attach all drives to a single SCSI bus or if you attach two drives each to two different SCSI buses. Both configurations will usually have identical disk subsystem performance. This applies to applications such as database transaction processing, which generate random disk operations of 2 KB or 4 KB. The SCSI bus does not contribute significantly to the total time required for each I/O operation. Each I/O operation usually requires drive seek and latency times; therefore, the sustainable number of operations per second is reduced, causing SCSI bus utilization to be low.

For a configuration which runs applications that access image data or large sequential files, performance improvement can be achieved by using a balanced distribution of drives on the three SCSI buses of the ServeRAID.