Markov Decision Process Model

In order to keep the complexity of the problem manageable, only one channel resource, i.e., a single subcarrier, is considered at a time. In the following, the subcarrier index n is omitted in the denotation of the SNR γi,j and the channel transfer factor hi,j. For the MDP model of the considered problem, all relevant parameters need to be taken into account. For the considered problem, a state s is defined by the following parameters:

• The data buffer level Bi of each forwarding node i of the cluster.

• The channel conditions of the nodes, more precisely, the number of data packets N_p,imax that each node i is able to transmit at most in the current time slot. This is only one value per node, since only one subcarrier is considered and the nodes always transmit to the strongest receiver.

Since the aim is to maximize the throughput of data packets, a transmitter always adapts its transmission rate to the strongest available receiver. Therefore, for each forwarding node i, only the strongest channel SNR is of interest, which is denoted by γi,max = max

j γi,j.

An action a refers to the allocation of the channel resource to a certain node. The reward Ra

ss0 incurred by taking action a in the state s and leading to state s0

is given by the number of data packets that can be transmitted by choosing this action.

The achievable data rates in practice depend on various parameters concerning the devices and the transmission protocol. Each device performs differently, for instance,

48 Chapter 4: Resource Allocation in Corridor-based Routing

Table 4.1. Rate adaptation

SNR capacity Nmax p / time slot < 4.8 dB < 2 bit/s/Hz 0 4.8 - 11.8 dB 2 bits/s/Hz 1 11.8 - 18 dB 4 bits/s/Hz 2 > 18 dB 6 bits/s/Hz 3

in terms of the receiver sensitivity. How many data packets can be transmitted de- pends on, for instance, the data packet size, the modulation and coding schemes, the carrier frequency and so on. The minimum required SNR to transmit data packets at all is usually in the range of 2-8 dB for most of the 802.11 protocols [ZTZ+08], [YLQ+17], [HLLS04]. In order to avoid any assumptions concerning specific param- eters and to keep the investigation as general as possible, the following mapping is proposed. Without loss of generality, 2 bits/s/Hz are assumed as a minimum required capacity to transmit a single data packet within a time slot. This corresponds to a minimum SNR of 4.8 dB. It follows that 4 bits/s/Hz are the required channel capacity to transmit 2 packets per time slot, which corresponds to a minimum SNR of 11.8 dB and so on, as shown in Table 4.1. This mapping captures the general functioning of an adaptive selection of the Modulation and Coding Scheme (MCS) for transmission. Usually, there is a limited amount of MCS options available that can be applied for transmission. Each MCS results in a certain data rate.

The final step to complete the MDP model is to find the transition probabilities be- tween the states P_ssa0 = Pr{st+1 = s0|st = s, at = a}, where st denotes the initial state in time slot t and st+1 denotes the resulting state in time slot t + 1. Since the number of data packets in each data buffer and the number of data packets that can be trans- mitted by choosing action a are known, the data buffers levels in the resulting state are also known. The only unknown parameters are the channel conditions in the upcoming state. Of course, the actual upcoming channel states are unknown, but channel statis- tics, i.e., the average SNRs, are known. Furthermore, Rayleigh fading is assumed on the channels which means that the real and the imaginary part of a received signal are independent normally distributed variables with a variance of σ2_i,j. The magnitude of a signal transmitted by node i and received by node j follows a Rayleigh distribution and is given by

Ai,j = √

4.4 Resource Allocation 49

The Probability Density Function (PDF) of Ai,j is given by

pdf(Ai,j) = 2Ai,j σ2 i,j e −A2i,j σ2_i,j , for Ai,j ≥ 0, (4.2)

The corresponding Cumulative Distribution Function (CDF), which gives the proba- bility that Ai,j lies below or is equal to a certain value, is given by

cdf(Ai,j) = 1 − e −A2i,j

σ2

i,j, for A

i,j ≥ 0. (4.3)

The PDF of the signal with maximum magnitude Ai,max = max j

√

psc· |Ai,j| is given by the probability that one magnitude is equal to a certain value while the magnitudes of the remaining signals are below that value

pdf(Ai,max) = Ncn X j1=1 pdf(Ai,j1) Ncn Y j2=1, j26=j1 cdf(Ai,j2) = Ncn X j1=1 2Ai,max σ2 i,j1 e −A2i,max σ2 i,j1 Ncn Y j2=1, j26=j1  1 − e −A2i,max σ2 i,j2  . (4.4)

In Figure 4.4, an example with three Rayleigh channels with an average SNR of ¯

γi,1 = 10 dB, ¯γi,2 = 12.5 dB and ¯γi,3 = 15 dB is illustrated. The individual PDFs are shown in Figure 4.4 a). The resulting PDF of Ai,max = max

j √

psc· |hi,j| is given in Figure 4.4 b). The resulting Probability Mass Function (PMF), which gives the number of transmittable packets N_p,imax assuming the mapping according to Table 4.1, is shown in Figure 4.4 c).

Using the known average SNR information of the current stage, the PMF for the number N_p,imax of transmittable packets for each forwarding node i in the cluster can be determined. Based on this, the transition probabilities between the states are given by the combination of these probabilities. Since the individual channel states are assumed to be independent of each other, the probability of a certain combination of channel states is given by the multiplication of the individual probabilities.

4.4.1.3 Optimal Resource Allocation Policy based on Dynamic Program-