4.3 Solution Method and Analysis when Weights are Known
4.3.3 Simulation Study
This section will give the results of simulation experiments investigating the sets chosen by each of the algorithms from Section 4.3.2 under the different click models. The arm sets selected by different algorithms will be analysed first for set CTR, then for set diversity.
Results are based on 1000 instances. Each instance represents a unique user. Each weight is drawn for each instance independently from a mixture distribution where each is relevant with probability ξ = 0.5 and non-relevant otherwise. If relevant the weight is drawn from a Beta(α, β) distribution, otherwise the weight is 0.001. The values of the parameters α, β will be given with the results. The low non-relevance weight represents a mismatch between the element and the user’s state where it is assumed that the user will click with some low non-zero probability. On each instance, the state distributionqis sampled from a Dirichlet distribution with alln parameters equal to 1/n. In response each set choosing algorithm selects a set of m arms from the availablek. In all simulations there are n= 20 possible states for x with k = 40 elements and a set size ofm= 3. Varying these parameters did not change the overall pattern of results.
Experimental Results: CTR
Table 4.3.1 gives the percentage lost CTR which is the the difference between the CTR of the sets chosen by the OPT and those chosen by the heuristic as a percentage of the optimal CTR. This is averaged over all instances. In each instance there is a
CHAPTER 4. SELECTING MULTIPLE WEBSITE ELEMENTS 88
true click model (either PCM or TCM, as given in the left column) and a set choosing algorithm. The click model assumed by OPT or SEQ is appended to its name (e.g. SEQ-TCM assumes that the click model is TCM - whether that is correct or not). The other methods do not assume any click model. The absolute CTRs of OPT using the correct click model with β = 2 and β = 9 respectively are 0.877 and 0.451 for PCM, and 0.773 and 0.316 for TCM.
Click
β Set Choosing Method
Model OPT-PCM OPT-TCM SEQ-PCM SEQ-TCM NAI MFUP OUP
PCM 2 0% 9.2% 0.0% 8.9% 4.2% 17.0% 9.8%
PCM 9 0% 19.9% 0.0% 19.8% 0.6% 13.6% 20.7%
TCM 2 3.4% 0% 3.4% 0.1% 10.6% 25.6% 1.1%
TCM 9 6.6% 0% 6.6% 0.1% 9.6% 27.2% 1.4%
Table 4.3.1: Lost reward as a percentage of optimal reward over 1000 instances with
n= 20, k = 40, m= 3, ξ = 0.5,α = 1 andβ as shown.
It can be seen that SEQ performs similarly to OPT with performance being much better than the theoretical guarantees when the assumed click model is correct. How- ever, both methods do badly when the click model is incorrectly specified. The other methods perform poorly on at least one of the click models with OUP better on TCM and NAI better on PCM. These preferences can be explained by the set diversity for each as will be given in the next section.
The performance of NAI illustrates an issue with PCM as a choice for click model. Despite NAI ignoring interaction effects, for PCM withβ = 9 it performs well. When
wa is small, rPCM(x, A,wA) = 1−Qa∈A(1−wa,x)≈
P
a∈Awa,x, and NAI is optimal
for the problem of maximising the expected value of this last quantity. So if weights are small (as would be common in problems such as web advertising) then using PCM does not result in sets where interactions between elements are important. This goes against the intuition that interactions are important.
CHAPTER 4. SELECTING MULTIPLE WEBSITE ELEMENTS 89
Experimental Results: Diversity
To compare set diversity of different methods and click models we first need a mea- sure of diversity appropriate to our problem. From Vargas and Castells (2011), the “pairwise dissimilarity between recommended items” is commonly used. Based on this we define the following simple measure of redundancy or overlap:
Definition 4.3.3 (Overlap and Diversity). The overlap of a set of two arms A =
a1, a2 is overlap(A) = Pn x=1min(w1,x, w2,x) min[Pn x=1(w1,x), Pn x=1(w2,x)] .
For sets of arms larger than 2 the overlap is given by
2
|A|(|A| −1)
X
ai,aj∈A,i<j
overlap({ai, aj}).
The diversity of the set A is given by 1−overlap(A).
Note that these measures are independent of the click model used. For two arms, if overlap = 1 then the weights for one arm are all larger than the corresponding weights of the other arm. In that case the lesser of the two arms contributes nothing under TCM. However, it does contribute under PCM. Ifoverlap= 0 then the weight vectors of all arms are pairwise orthogonal and CTRd(A) = P
a∈ACTR
d(a) for all
CHAPTER 4. SELECTING MULTIPLE WEBSITE ELEMENTS 90
Figure 4.3.1: Overlap for set choosing algorithms over d ∈ [1,8] averaged over 1000 instances with m= 3, n = 20, ξ= 0.5 andk = 40 arms, each with α= 1 andβ = 2.
To investigate diversity we measure the mean overlap values of element sets chosen by algorithms. These are given in Figure 4.3.1 over a range of GCM parameter values
d used by the OPT and SEQ algorithms. The other methods ignore the click model and so have unchanging overlap overd.
The overlap of the optimal sets decreases withdwhich shows how the diversity require- ments change with the click model. SEQ shows a very similar pattern and chooses similar sets to OPT but with generally slightly greater overlap.
NAI and MFUP choose sets that are insufficiently diverse for any model which explains their poor performance in the CTR simulation results particularly on TCM. OUP has low overlap which is at an appropriate level for TCM and higher d but which is too low for PCM which fits with its poor CTR on PCM. No method is as diverse as OPT and SEQ for high d indicating that maximising rewards under the TCM model is an effective method to produce diverse sets.
CHAPTER 4. SELECTING MULTIPLE WEBSITE ELEMENTS 91