Confidence Interval

The use of the confidence interval as a means of demonstrating that performance metrics have been estimated with high degree of confidence is discussed in Chapter 3 and applied to validate the simulation results in this thesis. In this Chapter, it is used in a different manner to ensure sufficient traffic statistics are collected to estimate QoS and also to regulate the time adaptation decisions are made. Specifically, the confidence interval is used for estimating the mean of the carried traffic and also for the blocking probability. While, the mean of the carried traffic is assumed to have a normal distribution, the blocking probability is assumed to have a

binomial distribution since it involves binary outcomes i.e. failure or success. Therefore, confidence intervals relating to normal and binomial distributions are discussed in the following.

The confidence interval estimates a population parameter (such as mean) with an interval and it is usually accompanied by a confidence level [136]. The confidence level (also referred to as degree of confidence) is the probability that the population parameter falls within the confidence interval [137]. For a random sample, X1, X2,

…, Xn, taken from a population with an unknown mean, µ, and known variance, ,

the confidence interval evaluated with a confidence level of for the mean is such that [136]:

(6.1) and are random variables of the lower and upper limits of the interval estimate for the mean respectively and .

If the sample considered is large (n ≥ 30) and it has a point estimate of mean given by , then has an approximate normal distribution with mean, , and variance, [138]; the confidence interval limits can be obtained starting with standardizing as follows as shown in [138]:

(6.2)

Z has an approximately standard normal distribution. For any given Z, there is a value (which is the upper 100 ( percent point of the standard normal distribution [136]) such that

= (6.3)

Substituting (6.2) in (6.3) and expressing inequalities relative to instead of Z gives:

= (6.4)

If are the values of the random variables and respectively, then the lower and upper confidence limits ( and ) are given as follows:

(6.5)

(6.6)

For the large sample case, if the variance of the population, is unknown it can be replaced by the estimated variance, s, of the sample, without a significant effect on the accuracy [138].

can be determined from (6.3) and the graphical representation of the probability on the left hand side (LHS) of the equation. Since Z is a standard normal distribution it has a mean of 0 and the area under the associated integral is evenly distributed into symmetric negative and positive sides as shown in Figure 6.5. Therefore, the LHS of (6.3) can be expressed as the summation of two probabilities, one for the positive side and the other for the negative side of the area of the integral:

= + (6.7) 2 α z + 2 α z −

Figure 6.5 Standard Normal Curve for

due to the symmetry of the positive and negative areas:

(6.8) Hence, from (6.7)

(6.9)

Since is known, can be evaluated from a standard normal integral table which gives numerical values for integrals of the form [138]:

మ

(6.10)

Four confidence levels (30%, 60%, 90% and 99.9%) are considered in this work to demonstrate the impact of quick decision making and sample size of collected statistics on the performance of the algorithm. The confidence levels considered here also include low confidence levels unlike in Chapter 3 where typical high confidence levels used in confidence interval estimations are provided. The associated value for each confidence level is shown in Table 6.1.

Table 6.1 Confidence Interval Parameters

Confidence Level 30% 60% 90% 99.9%

0.39 0.84 1.65 3.29

The maximum deviation from the point estimate of mean, , based on the

confidence level is used in this work to decide when the average carried traffic has converged. The estimated maximum deviation for average carried traffic is represented by and from the expressions for the lower and upper confidence limits, (6.5) and (6.6) respectively, is given by:

(6.11)

is used as criterion for determining that the average carried traffic has converged , where is a real number and is a desired maximum deviation threshold for deciding the carried traffic convergence. This implies that the maximum deviation must be lower than the prescribed value, , at the specified confidence level, , before it is accepted that the average carried traffic has converged. The use of the confidence interval for convergence determination is hinged on the fact that the traffic statistics are collected over time and the sample size grows with time rather than all sample sizes being available at any time. Hence, after a transition time such as a policy level change, initial individual carried traffic entries collected will vary during this unstable initial phase and confidence intervals will be wide due to the high variance. This is because the estimated maximum deviation, , is directly proportional to the standard deviation (which is the square root of the

variance) as can be observed from (6.7). However, as the system stabilises by adjusting to the new policy level, the variation in the carried traffic will reduce and so will the confidence intervals estimated.

Also from (6.7) and the values of for the different confidence levels from Table 6.1, for a given value of , the higher the confidence level the higher the sample size that will be required to satisfy the criterion. This is because the value of increases with the confidence level and the estimated maximum deviation, , is directly proportional to while it is inversely proportional to the sample size, . Since, the traffic statistics sample size increases as time passes; then, higher confidence levels will lead to longer decision epochs but will produce estimates with better degree of confidence.

As mentioned earlier, the confidence interval for the binomial distribution is considered to handle the case of blocking probability. The binomial distribution is characterised by a parameter, , which is the probability of success and the number of trials, n [139]. Only two outcomes are possible for each trial, i.e. failure or success. Assuming that is the probability of success estimated from a large sample

(n ≥ 30), the confidence interval for at a confidence level of can be

estimated in a similar way as the mean, , for the population considered earlier as follows [138]:

(6.12)

This is the case because the random variable, with outcome , is an unbiased estimator of and has a normal distribution with mean and variance,

[139].

The blocking probability evaluated from the data collected from the different zones is defined in a similar way as in this study, with blocking considered as the successful event being counted. Also, like the carried traffic case, the maximum deviation criterion has to be satisfied before decisions are taken. However, in this case, the focus is on the use of the criterion to estimate the blocking probability with

some degree of confidence not just convergence. The estimated maximum deviation for blocking probability, , is given by:

(6.13)

The criterion in this case is given by , where is the desired maximum deviation threshold for blocking probability. How this criterion is applied in adapting the RRM and TM parameters to traffic load are discussed in detail under the algorithm implementation in the next section.