This section will walk you through actual data transfer using the offload programming language. In Code Listing 6-1, I show how you can use the offload programming language to set up a high-level data transfer between hosts to Intel Xeon Phi. Although there are lower level APIs exposed by Intel Xeon Phi programming environments available through Intel Xeon Phi system programming software, such as the symmetric communication interface (SCIF), I will focus on high-level offload language extensions for C to do the data transfer. The Intel compiler uses appropriate lower-level system interfaces for such tasks as setting up the DMA channel for proper transfer and so forth. You will see, however, there are some requirements for optimizing data transfer performance.
2For details of the descriptor rings format, see Intel Xeon Phi Coprocessor System Software Developers Guide. IBL Doc. ID: 488596.
Chapter 6 ■ Xeon phi pCie Bus Data transfer anD power ManageMent
Code Listing 6-1. Example of Host to Device Data Transfer Over the PCIe Bus 38 //Define number of floats for 64 MB data transfer
39 #define SIZE (64*1000*1000/sizeof(float)) 40 #define ITER 10
41 // set cache line size alignment 42 #define ALIGN (64)
43 __declspec(target(MIC)) static float *a; 44 extern double elapsedTime (void); 45 int main()
46 {
47 double startTime, duration; 48 int i, j; 49 50 //allocate a 51 a = (float*)_mm_malloc(SIZE*sizeof(float),ALIGN); 52 53 //initialize arrays 54 #pragma omp parallel for 55 for (i=0; i<SIZE;i++) 56 {
57 a[i]=(float)1.0f; 58 }
59 // Allocate memory on the card
60 #pragma offload_transfer target(mic) \
61 in(a:length(SIZE) free_if(0) alloc_if(1) align(ALIGN) ) 62
63
65 startTime = elapsedTime(); 66 for(i=0; i<ITER;i++) {
67 //transfer data over the PCI express bus 68 #pragma offload_transfer target(mic) \
69 in(a:length(SIZE) free_if(0) alloc_if(0) align(ALIGN) ) 70
71 }
72 duration = elapsedTime() - startTime; ; 74 // free memory on the card
75 #pragma offload_transfer target(mic) \
76 in(a:length(SIZE) alloc_if(0) free_if(1) ) 77
78
80 //free the host system memory 81 _mm_free(a);
83 double GB = SIZE*sizeof(float)/(1000.0*1000.0*1000.0); 84 double GBps = ITER*GB/duration;
85 printf("SP ArraySize = %0.4lf MB, ALIGN=%dB, PCIe Data transfer bandwidth Host->Device GB/s = %0.2lf\n", GB*1000.0, ALIGN, GBps);
86
87 return 0; 88 }
Chapter 6 ■ Xeon phi pCie Bus Data transfer anD power ManageMent Recall that the maximum data transfer performance can be achieved by using the DMA engine. Since DMA engines transfer data in 64 bytes (cache line size), you need to make sure the data length is a 64-byte multiple. At Line 39, I choose the data array size to transfer as 64MB. I chose a large size to amortize the DMA transfer overhead needed to set up the DMA engine with proper parameters. The second important datum of information needed to optimize the transfer is to make sure that the data to be transferred are cacheline-aligned to enable transfer of the data with the DMA engine. I used the ‘C’ macro ‘#define ALIGN’ in Line 42 to make sure the allocated memory is 64-byte aligned. In Line 43, I define a pointer ‘a’ that will be visible to offloaded code executing on the card by declaring it with “__declspec(target(MIC)).”
This pointer will be allocated on the host and transferred over to the coprocessor using offload pragmas. In order to do that, you need to allocate space for the buffer using _mm_malloc intrinsics supported by the Intel compiler in Line 51. The _mm_malloc intrinsic, in addition to behaving like a malloc function, also allows data alignment to be specified as part of the request. This line allocates 64MB of data area aligned to the cache line boundary (64 bytes).
The first data transfer happens in Line 60, as shown below: 60 #pragma offload target(mic) \
61 in(a:length(SIZE) alloc_if(1) free_if(0) align(ALIGN) ) 62 {
63 }
The data transfer begins by a call “#pragma offload target(mic)” that tells the runtime library to start a process started on coprocessor. The ‘in’ parameter tells it to send the array pointed to by ‘*a’ to the coprocessor, the alloc_if(1) clause in the offload statement allocates memory on the card for pointer ‘a’ and copies the data over to coprocessor. Since the size of the buffer and alignment is large enough, the underlying libraries will use the DMA engine to set up and transfer these data to the coprocessor. The free_if(0) clause tells the runtime not to free the buffer that was allocated on the card so it could be reused. The ‘align(ALIGN)’ clause in the same statement tells the buffer that is allocated on the card to be aligned to ALIGN bytes, in this case 64 bytes.
I have separated out the first offload pragma call as it involves start-up overhead that I do not want to count as part of the transfer time. The overhead involves sending the coprocessor part of the binary to the card and sending necessary runtime compiler library like openmp library to the card as well. You will also need to allocate space for array ‘a’ on the card to copy the data in, which happens during this first offload statement.
Lines 65 through 72 are the timing loop where the actual data transfer rate is measured. Here the data transfer is repeated ITER (10) times to average out the any run to run variation in transfer time. The ‘pragma offload’ statement is similar to what was discussed earlier to show the initialization of coprocessor code and data: 68 #pragma offload target(mic) \
69 in(a:length(SIZE) free_if(0) alloc_if(0) align(ALIGN) ) 70 {}
The main thing to keep in mind is that in this statement we have both the ‘free_if(0)’ and ‘alloc_if(0)’ clause set to zero, thus telling the runtime not to allocate any more space for ‘a’ and also not to free the space created previously with pragma offload code. Note that if you made the previous statement to free the buffer allocated in the first call by calling with free_if(1), the buffer would have been freed after the ‘pragma offload’ statement.
In that case you would need to reallocate the buffer with the alloc_if(1) call instead of alloc_if(0) call. It is a common source of error in offload programming if the allocation and free clauses do not properly match up. If you have more ‘alloc_if(1)’ calls without matching free_if(1) calls, you will see memory leaks on the coprocessor card. Similarly, if you try to copy the input buffer with the alloc_if(0) call assuming the buffer is present, but where it was accidentally freed in the previous call, you will get a general protection fault.
In Line 75, I am done with the data transfer measurements and have freed the memory allocated on the card by using the free_if(1) call with a matching alloc_if(0) call so that I do not need to allocate the buffer on entering the offload region.
Chapter 6 ■ Xeon phi pCie Bus Data transfer anD power ManageMent
Figure 6-3a. Output of pciebw test run
The Intel compiler recognizes offload pragmas and generates the code necessary for data transfer to the coprocessor.
The output captured on Figure 6-3a shows that the data transfer of 64 MB buffer size with 64-byte aligned achieved an approximately 6.8 GB/s transfer rate over the PCIe bus in moving from the host to the coprocessor (device). You can also see the effect of various buffer sizes and alignment if you play around with the alignment and data buffer size. To see the effect of various buffer sizes, let’s modify the code to run with 64 kB instead of 64 MB. This can be done by changing the buffer size in Line 39 to:
39 #define SIZE (64*1000/sizeof(float))
I recompiled and ran the same code with the modified buffer size and the output is shown in Figure 6-3b. As you can see, the BW drops to 3.88 GB/s compared to 6.8 GB/s for the 64 MB buffer size. This is due to DMA transfer overhead, because smaller data transfer sizes cut down on overall performance.
Figure 6-3b. Output of the pciebw test with 64K data buffer
Now, to see what the effect of alignment is, let’s change the buffer alignment from 64 bytes to 6 bytes of unaligned data. As you can see from Figure 6-3c, the BW drops even further to 2.29 GB/s.
I also need to free the memory allocated on the host using _mm_malloc by matching _mm_free intrinsics. You need to make sure you use _mm_free instead of free calls since they use a different heap manager and a different space for buffer management. Figure 6-3a shows the output of “pciebw.out” application built by the following command with Intel compiler:
Chapter 6 ■ Xeon phi pCie Bus Data transfer anD power ManageMent
It has been found that for smaller data sizes (< 4 KB), the data transfer may be faster when it is done with MMIO through CPU write to card memory than through DMA transfer. The compiler may choose accordingly to use different data transfer methods depending on the data buffer size for optimal data transfer bandwidth. You could run the same example with OFFLOAD_REPORT=1. This will print out the CPU time taken to transfer the data and should correlate to the performance number for various alignment cases discussed in this section.