throughput per user (the throughput in Figure 7
divided by the number of users) .
Cl z 60 0 (.) 50 w en a: 40 w o._ en 30 z 0 t= 20 (.) ..: en 1 0 z ..: a: 0 f- 0 KEY: 0 VAX 8978 £ VAX 8974 .t;. VAX 8700 200 400 600 CPU UTILIZATION
Figure 9 Throughput versus CPU Utilization
87
VAXcluster Systems
System Level Performance of VAX 8974 and 89 78 Systems 200 400 600 800 1 000 1 200 1 400 NUMBER OF USERS KEY: 0 VAX 8978 J1. VAX 8974 !!. VAX 8700
Figure 1 0 User Productivity
This figure shows that the maximum through put per user for this workload is around 1 50 TPS for any configuration . This graph also indicates the number of users that can be supported by each system while mainraini ng a certain level of user productivity. For example, at 1 4 0 TPS , the 8700, 8 9 7 4 , and 8978 support 2 5 0 , 8 5 0 , a nd 1 , 2 00 users respectively. More users can be sup ported at lower user productivity levels.
Figure 1 0 also indicates the level of users at which one m ight consider switching to a larger system to maintain a certai n l evel of user pro ductivity . For example , to maintain a user pro ductivity level of approximately 1 5 0 TPS, one must switch to a VAX 8974 system at around 2 4 0 users, and to a VAX 8978 system at around 7 2 0 users.
Mean Service Time
The VAX 8700 and VAX 8974 service ti mes remained under one second for a l l user l evels tested. The VAX 8978 service-time curve a lso fol lowed this trend u p to the 960-user level . How ever, after that level, the service time degraded quickly due to the large number of IjOs and queue lengths at the disks as the 1 200-user level was approached . These patterns are shown i n Figure 1 1 .
ENQ Rate
So far, only user visible performance and some system behavior has been discussed . Now some of
88 2.50 2.00 (j) 0 1 .50 z 0 u 1 .00 w (j) 0 50 0 0 200 KEY: D VAX 8978 J1. VAX 8974 !!. VAX 8700 400 600 800 1 000 NUMBER OF USERS
Figure 1 1 WIC Service Time
1 200 1 400
the cl uster aspects of the systems are examined, mainly the locking activities.
As mentioned at the beginn i ng of this paper, the WIC workload assumes ful l data-sharing (i . e . , a l l the database fi les are shared by all users) . This sharing i nvolves locking and un locking fil es and records every time they are accessed . The locking and unlocking operations are performed by sys tem services called ENQ and DEQ . An ENQ request is serviced by the d istri buted lock man ager, which exa m ines outstand ing locks to the resource and a llows access if there is no confl ict .
The SPM software records the the number of ENQs on a particular processor. The total ENQ rates at different user levels for differem configu rations were extracted from SPM data and graphed in Figure 1 2 . This cu rve cl osely resem bles the throughput curve, implying a strong cor relation between locking activities and through put. Around 26 ENQ operations were required on the average to perform each exchange .
Total Remote ENQ Rate
A remote ENQ occurs when the resource of inter est is mastered by a process that runs on an other processor in the c luster. Remote locks are more costly than local locks because additional interprocessor commun ication over the CI bus is required between the requesti ng and mastering nodes.
Figure 1 3 plots the remote ENQ rates aga inst the tOtal ENQ rates for different configurations.
Digital Technical journal
200 400 600 800 1 000 1 200 1 400 KEY: 0 VAX 8978 • VAX 8974 D. VAX 8700 NUMBER OF USERS
Figure 12 Total ENQ Rate
The increasing slopes of the different curves ind icate that the remote EN Q rate also i ncreases with the nu mber of processors in the system as well as with the total nu mber of users. Generally, i n an N-processor homogeneous distributed sys tem i n which all resources are equally accessed by all processors and all accesses require locking operations, the remote locki ng operations wi ll equal (N - 1 ) /N times the total locking activity. This result occurs because each processor has an equal opportunity to master a particu lar resou rce. This relationship held in the case of t he remote versus the total new ENQ rates observed in the VAX 8974 and VAX 8978 systems, in which the ratios were 75 percent and 8 7 . 5 percent respectively. Figure I 3 shows, however, that on the average only 60 percent and 80 percent of the ENQs were remote for the 8974 and the 8978 respectively. These resu lts occu rred because t he ploned ENQ rate includes the convened ENQ rate as well as the new ENQ rate; most convened
ENQs were found to be loca l . lnterprocessor Com munication
The communications between the processors are achieved by the Systems Communication Archi teCture by way of transmitting and receiving sequenced messages. Figure 1 4 shows the num ber of sequenced messages transferred between the processors every second . Most of these mes sages are generated by t he distributed lock man ager for clusterwide locking purposes.
·Digital Technical journal No. 5 September I ')87 TOTAL ENO/SECOND KEY: 0 VAX 8978 • VAX 8974 D. VAX 8700
Figure 13 Remote versus Total ENQ Rates
0 z 0 frl 4 .00 � 3.50 (f) 0 3.00 z ;?, 2.50
6
2.00 I 1- 1 .50 I (f) 1 .00 w � 0.50 (f) l{l 0 ::2 200 400 600 800 1 000 1 200 1 400 KEY: D VAX 8978 • VAX 8974 D. VAX 8700 NUMBER OF USERSFigure 14 Message Rate between Processors
Cl Traffic
The traffic on the CI consists of three packet types: datagrams, sequenced messages, and block transfer messages. In this appl ication , datagrams were used only for error loggi ng and t herefore did not exist . Sequenced messages are used for communications between the processors and the HSC70 controllers . Most of these short packets are either packets between the d istri buted lock managers to perform clusterwide lock ing (dis cussed earl ier) or packets between a processor and an HSC70 controller to request and response to I/0 operations. Each 1/0 request to the disks
89
VAXcluster Systems
System Level Performance of VAX 89 74 and 89 78 Systems
or tapes controlled by an HSC70 device requires a pair of messages to be exchanged between the processor and the controller. Block transfer mes sages are data packets for 1/0 operations. The transfer rates of each message type are recorded by the SPM software . Figure 1 5 pl ots the CI traf fic against the n umber of users. The CI traffic, expressed i n KB per second , is calculated from the data collected by the SPM software.
This figure shows that, in general , the CI bus is rather underutilized , peak ing around I , 2 65KB per second at 1 , 200 users for the VAX 8978 sys tem . This utilization is less than I 5 percent of the raw bandwidth of a single CI wire , or 7 . 5 percent of the bandwidth on each CI path. It should be noted, however, that this data includes neither the extra bytes of the lower level protocol over head nor the additional traffic incurred by retransmissions. Thus the actual CI util ization will be a little higher than these figures.