2.2
Overview of parallel programming models
A parallel programming model can be regarded as an abstraction of a system ar- chitecture. Thus, the program written based on a parallel programming model can be compiled and executed for the underlying parallel architecture. Because of the wide range of existing parallel systems and architectures different parallel programming mod- els were created. A parallel programming model is general if the mapping of a wide range of problems for a variety of different architectures is efficient. The underlying memory architecture highly influences what parallel programming model is going to give a good abstraction. The three most fundamental parallel computer memory architectures are:
shared memory, distributed memory and hybrid distributed-shared memory.
In shared memory parallel computers the processors share the same memory re- sources, thus, there is only a single memory address space. Cache coherency is available, meaning that every memory update is visible to every processor. Shared memory com- puters can be further classified as: uniform memory access (UMA) and non-uniform memory access (NUMA) computers. The UMA model is implemented by symmetric multiprocessor (SMP) computers, where identical processors are placed near the shared memory, thus, the memory access time is equal to every processor. The NUMA model is implemented by linking several SMPs and every SMP can directly access the memory of another SMP. However, in this case the memory access time is different for processors lying in different SMPs. The advantages of this model are the global address space and fast data-sharing. The main disadvantages are the lack of scalability and the need of synchronization in order to ensure data correctness.
The distributed memory architecture model is encountered in networked or dis- tributed environments such as clusters or grids of computers. In this case every processor operates independently and has its own local memory, thus, there is no global memory address space. No cache coherency is present, so the memory change of one processor does not effect the memory of other processors. The programmer’s task is to manage the data communication and the synchronization of different tasks. The advantage of this approach is that each processor can rapidly access its own memory because no overhead is required to assure global cache coherency, and the memory can easily scale with the number of processors. Among the disadvantages of this model is the non-uniform memory access time and the burden to manage the data communication between processors.
Nowadays, systems implementing the hybrid distributed-shared memory model are the largest and fastest computers. In this model multiple shared memory processors are
2.2. OVERVIEW OF PARALLEL PROGRAMMING MODELS
connected through a network resulting in a distributed architecture. The hybrid model can be further improved by adding GP-GPUs to the shared memory architecture. This is a very powerful model, however, the programming complexity is significantly increased. Thus, in order to exploit the benefits of this hybrid model efficient algorithm mapping, computational load balancing and communication organization has to be designed.
The two predominant parallel programming models using pure shared or distributed memory approach are the Open Multi-Processing (OpenMP) [41] for shared memory and the Message Passing Interface (MPI) [42] for distributed memory systems. In addition, with the advent of the massively parallel many-core GP-GPUs new programming models like CUDA [40] and Open Computing Language (OpenCL) [43] were introduced.
OpenMP is a high-level abstraction of shared memory systems. OpenMP is imple- mented as a combination of a set of compiler directives, pragmas, and a runtime providing both management of the thread pool and a set of library routines. With the help of the directives the compiler is instructed to spawn new threads, to perform shared memory operations, to perform synchronization operations, etc. The fork-join threading model is used to switch between sequential and parallel operation modes. During a fork operation a thread splits into several threads, thus, the execution branches off in parallel, and after their task is completed they are joined resuming to sequential execution.
MPI is a parallel programming model for distributed memory systems and facilitates the communication between processes by interchanging messages. The communication is cooperative and occurs only when the first process executes a send operation and the second process executes a receive operation. The task of the programmer is to manage the workload by defining what tasks are to be performed by each process.
Since massively parallel many-core GP-GPUs are playing an important role in this thesis, in Sec. 2.2.1 the most important definitions and components of CUDA are pre- sented.
2.2.1 The CUDA programming model
The programming of GP-GPU devices has became popular since Nvidia published the CUDA parallel programming model. Traditional CPUs are able to execute only a few threads, but with relatively high clock rate. In contrast, GP-GPUs have a parallel architecture that support the execution of thousands of threads with a lower clock-rate. An extensive description of CUDA programming and optimization techniques can be found in [40]. The main entry points of GP-GPU programs are referred to as kernels.
2.2. OVERVIEW OF PARALLEL PROGRAMMING MODELS
These kernels are executed N times in parallel by N different CUDA threads. CUDA threads are grouped in thread blocks (TBs). The number of threads in a TB is limited, however, multiple equally-shaped TBs can be launched simultaneously. A grid is a col- lection of TBs. The threads in the TB or the TBs in the grid can have a one-dimensional, two-dimensional or three-dimensional ordering.
The cooperation between the threads is realized with the help of multiple memory spaces that differ in size, latency and visibility. In CUDA the following hierarchy of memory levels are defined: (i) private, (ii) shared, (iii) global, (iv) constant and (v) texture memory.
In some cases specific threads have to wait for the results generated by other threads. Therefore, threads within a TB can be synchronized. In order to continue the execution, each thread has to reach the synchronization point. There is no similar mechanism to synchronize TBs in a grid. When a kernel finishes its execution it can be regarded as a global synchronization of the TBs.
The Nvidia GP-GPU architecture is built around a scalable array of multithreaded streaming multiprocessors (SMs). The TBs of the grid are distributed to the SMs with available execution capacity by the grid management unit. An important metric of the SMs usage is occupancy. The occupancy metric of each SM is defined as the number of active threads divided by the maximum number of threads. Groups of 32 threads, called
warps, are executed together. The maximum number of TBs running simultaneously on
a multiprocessor is limited by the maximum number of warps or registers, or by the amount of shared memory used by the kernel.
In order to concurrently execute hundreds of threads, the SMs employ a SIMT archi- tecture. A warp executes one common instruction at a time. In the case of branching, the warp will serially execute each branch path. In order to achieve full efficiency, divergence should be avoided. Applications manage concurrency through streams. A stream is a sequence of commands that are executed in order. Different streams may execute their commands out of order with respect to one another or concurrently. Thus, launching multiple kernels on different streams is also possible. This can be very efficient when kernels can be launched independently from each other.
Chapter 3
Overview of MIMO
communications
3.1
Benefits of MIMO systems
The key feature of MIMO systems is the ability to turn multipath propagation, traditionally a pitfall of wireless transmissions, into a benefit for the user, thus, the performance of wireless systems is improved by orders of magnitude at no cost of extra spectrum use. The performance enhancements of MIMO systems are achieved through
array gain, spatial diversity gain, and spatial multiplexing gain.
Array gain is defined as the efficient combination of the transmitted or received signals that result in the increase of the signal-to-noise ratio (SNR). As a result the noise resistance and the coverage of a wireless network is improved. Array gain is exploited by methods like: selection combining, maximal-ratio combining, equal-gain combining as discussed in [44].
In case of spatial diversity different representations of the same data stream (by means of coding) is transmitted on different parallel transmit branches, i.e., it introduces controlled redundancy in both space and time. At the receiver side independent copies of the same transmitted signal are encountered because of the rich scattering environment. If a signal path experiences a deep fade the original signal still can be restored, thus, the probability of error is minimized. The number of copies received is referred to as the diversity order. A MIMO channel with n transmit antennas and m receive antennas potentially offers n · m independent fading links, and hence a spatial diversity order of
n · m. Space-time coding (STC) was introduced to support transmit diversity in [45],
[46], [47], [48].
3.2. MIMO SYSTEM MODEL + + + + + +
Figure 3.1: MIMO system architecture with n transmitter and m receiver antennas.
Spatial multiplexing focuses on maximizing the capacity of a radio link by transmit- ting independent data streams on different transmit branches simultaneously and within the same frequency band. In a rich scattering environment, the receiver can separate the data streams and since each data stream experiences the same channel quality as a single-input single-output (SISO) system, the capacity is increased by a multiplicative factor equal to the number of streams. Spatial multiplexing was exploited in the following works [129], [49], [8].
In hybrid MIMO configurations the benefits of spatial multiplexing and space-time coding schemes are combined. This is achieved by dividing the transmit antennas into sub-groups where each sub-group is space-time coded independently. At the receiver group spatial filters are used followed by space-time decoding. For a few good examples refer to [50], [51] and [52].
3.2
MIMO system model
A MIMO system consists of n transmit and m receive antennas as shown in Fig. 3.1. The transmit antennas are sending a complex signal vector ˜stof size n during one symbol
period. The received complex symbol vector ˜y = (˜y1, ˜y2, ..., ˜ym)T is expressed as
˜
y = ˜H˜st+ ˜v (3.1) where ˜v = (˜v1, ˜v2, ..., ˜vm)T is the additive channel noise and the superposition of the
transmitted symbols is modeled by the channel matrix ˜H ∈ Cm×n. The statistical prop- DOI:10.15774/PPKE.ITK.2015.010