The Linux utility strace was used to gain more information about the system calls, in- cluding the size of each I/O call and the time taken for the system calls.
3.2
Baseline read performance
Baseline performance was investigated using C and Python reading from plain binary files for each system in the following sections.
JASMIN
Figure 3.4 shows the plain binary read performance using C and Python on the Panasas plat- form. The Python rate is lower at higher read buffer size (above 16MiB) than the C results - 30% at 1GiB buffer size. The profiles of the striding read with C and Python are very similar throughout the whole range of read buffer sizes starting off with very low read rates, increasing to around 400MB/s at 512MiB . The variability of the C reads is much higher than the python reads from 128MiB and above. The variability for the remaining read buffer sizes for each re- mains low. The random read profile increases in a similar way to the hopping reads, although with much higher variability when using C. It is not clear what has caused this variability.
The Python striding and random test were not run on the other platforms because of the general similarity between the C read rate and the python read rate from plain binary files on JASMIN, and for the sequential reads on the other platforms, only the C program was used to measure striding and random reads – because of the very long run times at such low buffer sizes.
ARCHER
The results for the plain binary reads on the Lustre platform are shown in Figure 3.5. The results are similar to the Panasas results, differing at low buffer size for the random reads with a more gradual slope up to peak performance, and an increase in performance above 64MiB for the sequential reads. The striding reads also have a higher read rate than would be expected from the JASMIN results above 256KiB.
Chapter 3. NetCDF4 Performance 3.2. Baseline read performance
Figure 3.4: Average read rate (diamonds) and one standard deviation (error bars) for the tests reading from a binary file on the Panasas platform. Grey lines show the results using C and black lines show the results using Python. The first graph shows the sequential reads, second shows hopping reads, and bottom shows random reads.
(a) (b)
(c)
Figure 3.5: Results from ARCHER showing the read rate from the C scripts reading from plain binary files
RDF
The results for the C plain binary reads on GPFS are shown in Figure 3.6. The profile for the sequential reads is similar to the Panasas results, albeit at higher bandwidth because of the
Chapter 3. NetCDF4 Performance 3.2. Baseline read performance Infiniband used on the GPFS platform compared to 10Gb/s Ethernet in Panasas. The vari- ability for the hopping reads is high which makes the profile more difficult to interpret. It is interesting however that the average striding reads are consistently higher than the random reads.
(a)
(b)
(c)
Figure 3.6: Results from the RDF using a C program to read from plain binary files.
Discussion
All of the results between different platforms have similar characteristics. At low buffer size for the sequential results on JASMIN, below 4-8KiB, there is a significant hit in performance; likely due to the reads being smaller than disk sector size which reduces read rate, although this has not been verified. For all platforms the sequential reads are the fastest, which is expected because it is a contiguous read on disk, and being the easiest for the file system to anticipate
Chapter 3. NetCDF4 Performance 3.2. Baseline read performance All the random reads and striding reads have similar profiles, showing the same gradual increase in read rate and have a lower read rate, interpreted as the effect of having to seek through the file. Because of the faster reads at larger buffer sizes, the effect of this is reduced due to larger reads. On JASMIN the small difference between the random and striding reads show that the size and direction of the seek does not have much of an effect on the read rate. The variation of the random reads is much higher than the other reads; likely due to the possi- bility of hitting caches because of the random nature of the read. The small difference between the striding and random reads and the shape of the profile implies that the size of the read has more of an impact in the read rate than the direction or size of the stride.
The results here compare well with that of others: the profile shape for the sequential reads for all platforms agrees with the results from Bartz et al. (2015). The expected bandwidth on the Lustre platform is around 500MB/s (Henty et al., 2015) which agrees with our results, and the peak read rate on JASMIN is similar to performance results (personal communication, Bryan Lawrence). The variation in the non-sequential reads compared to the sequential reads is also seen in Schmid and Kunkel (2016).
For all of the results there are two stand out patterns with increasing read size. The first is for the sequential reads which very quickly rise to a level maximum and stay roughly constant for the remainder of the read size, this shape will be referred to as a flat profile. The second pattern seen in the striding and random reads more gradually increases, peaking at larger read sizes, this will be referred to as a steadily increasing profile. Using these two terms, the results are summarised in Table 3.1.
The main conclusions from this section are as follows:
• The different read patterns have a very large effect on the read rate, with a lesser effect at larger buffer sizes. The cause of this is likely to be because of seeking through the file, which does not happen to the same extent in the sequential reads, therefore not decreasing the read rate. Alternatively, the file system could be successfully reading ahead for the sequential reads, but not for the striding and random.
• The read rate drops significantly at very small buffer size which agrees with results from Bartz et al. (2015).
Chapter 3. NetCDF4 Performance 3.3. NetCDF4 read performance