The software proposals done by this dissertation (Chapters 6 and 7) target NVIDIA GPUs (i.e., accelerators) and the CUDA programming model. The execution and memory model of NVIDIA CUDA is quite similar to the refer- ence programming model previously described. However, kernel invocation in NVIDIA CUDA uses different semantics. For instance, Listing 3.2 shows an example of a kernel call in CUDA. Kernel calls in CUDA take two additional parameters (<<<dimGrid, dimBlock>>>) to set up the thread organization ex- ecuting the kernel.
The specification of CUDA covers concepts ranging from the execution model on GPUs to the interface between CPUs and GPUs. We refer the reader to the NVIDIA CUDA documentation for a more comprehensive description of this programming model [NVI09]. Only the specific concepts that are used in the rest of the paper will be highlighted in this section.
3.3.1
Multi-GPU Support
CUDA allows applications to execute kernels in several GPUs through CUDA contexts. A CUDA context effectively represents a GPU device that is binded to an application execution thread. Kernel invocations and memory copy oper- ations performed by the application thread will be done over the device asso- ciated to the CUDA context. Application threads can request the creation of new contexts, but CUDA imposes the following constrains:
Processor Memory Subsystem
Freq: 5GHz L1 ICache: 16K (4-way, 1 port) Fet/Iss/Ret width: 4/4/5 L1 DCache:16K (4-way, 1 port) LdSt/Int/FP units: 4/4/3 L1 Hit/Miss Delay: 2/13 RAS: 32 L2 Cache: 1MB (16-way, 2 ports) BTB: 2K entries, 2-way assoc. L2 Hit/Miss Delay: 10/105 Branch pred: ITLB entries: 64 (4-way) bimodal size: 16K entries DTLB ent: 64 (4-way) gshare-11 size: 16K entries Main Mem: 271 I-window: 92 Acc Mem Delay: 271 ROB size: 176 Acc Link Channels: 16 Int regs: 96 Acc Link Latency: 1 FP regs: 80
Ld/St Q entries: 56/56 IMSHR/DMSHR: 4/16
Table 3.1: Simulation parameters. Latencies shown in processor cycles repre- senting minimum values.
• An application thread can only have one active CUDA context. When- ever an application thread attaches a new CUDA context, the old context becomes inactive.
• A CUDA context can only be active in one application thread. Attaching a CUDA context that is attached to other application thread will fail.
3.3.2
CUDA Streams
CUDA supports concurrent execution of kernels at the GPU and DMA transfers from and to GPU memory by means of streams. A CUDA stream is a sequence of commands that execute in order, but different CUDA streams might execute their commands out of order with respect to one another or concurrently. Thus, for kernel execution and DMA transfers to happen simultaneously, they must belong to different CUDA streams. Applications might overlap kernel execution, data transfers from system to GPU memory and from GPU to system memory using three streams. In a multi-GPU system, a CUDA stream is associated to the CUDA context where was created. Hence, all CUDA streams that belong to the same CUDA context will execute all actions (i.e., kernel calls and data transfers) in the GPU associated to such CUDA context.
3.4
Evaluation Methodology
3.4.1
Simulation Environment
Hardware modifications proposed in Chapter 5 are evaluated using execution- driven simulations. The architecture of the simulated CPU is shown in Table 3.1. The simulated DMA controller can read data from the L2 cache if present and from main memory otherwise. DMA transfers use burst reads and writes of 128 bytes in memory blocks of up to 4KB.
The evaluation is performed using a cycle-accurate simulator based on SESC [RFT+10],
System CPU GPU
Processor Memory Processor Memory
GTX285 2x AMD Opteron 4x 2 GB 2x GTX285 2x 1 GB 2222 3 GHz DDR2 30 SM GDDR3 2MB L2 667 MHz 1476 MHz 1242 MHz C870 2x Intel Xeon 4x 2 GB 2x C870 2x 1.5 GB E5420 2.5 GHz DDR2 16 SM GDDR3 6MB L2 667 MHz 1350 MHz 800 MHz
Table 3.2: Target systems used in the evaluation
hardware modifications proposed in this dissertation. The simulated CPU code is compiled using gcc version 3.4 with the -O3 flag to MIPS binaries.
Functional and timing simulation of the code executed by the CPU is per- formed. The code executed by the accelerator is natively executed in a NVIDIA GPU, so only CPU execution is considered. Total application execution time is estimated using the contribution of accelerator execution time to the to- tal application execution time. This contribution is obtained by executing the benchmark on the systems described in the following section.
3.4.2
Execution Environment
Experiments in Chapters 6 and 7 are executed in the 64-bit x86 systems outlined in table 3.2. All machines run a GNU/Linux system, with Linux kernel 2.6.32 and NVIDIA driver 195.36.15. Benchmarks are compiled using GCC 4.3.4 for CPU code and NVIDIA CUDA compiler 3.0 for GPU code.
Execution times are taken using the gettimeofday() system call, which offers a microsecond granularity. All experiments are executed 2048 times, and samples that are two times higher than the arithmetic average are considered outliers and discarded. All results use the arithmetic average value of the filtered results, and the standard deviation is used as an estimation of the error range. GTX285, in Table 3.2, is a commodity GPU system typically found in high- end desktop systems; C870, in Table 3.2, is a GPU system designed for high- performance environments. The GPUs in GTX285 are from a newer generation than the GPUs included in C870. Hence, some hardware features , such as concurrent DMA transfers and GPU execution, are only available on GTX285. Moreover, the peak performance of each GPU in GTX285 is 1062 GFLOPS, while each GPU is C870 reaches up to 518 GFLOPS. The peak performance accomplished by the systems has little impact over the experimental results presented in this dissertation because only data communication mechanisms are studied. These mechanisms mainly depend on the available bandwidth between system and accelerator memory in each system. Figure 3.5 shows the effective data transfer bandwidth between CPU (host) and GPU (device) for different data transfer sizes. Both systems accomplish a maximum data transfer bandwidth for large data sizes. Moreover, in both cases, there is a significantly hop in the effective accomplished bandwidth when the data transfer size goes
0 2 4 6 8 10 12 14 16 18 20 22 4KB 8KB 16KB 32KB 64KB 128KB 256KB 512KB 1024KB 2MB 4MB 8MB 16MB 32MB Gbps Size in bytes GTX285 Input Bandwidth
GTX285 Output Bandwidth C870 Output BandwidthC870 Input Bandwidth
Figure 3.5: GPU bandwidth for two different GPUs (GTX285 and C870)
from 1MB to 2MB. The available bandwidth in GTX285 is smaller than in C870, so the impact of CPU – accelerator data transfers in this system is larger.