Behavioral Ad Targeting

Applications such as behavioral ad targeting use browsing profiles to infer what the user is likely to be mostinterested in. For instance, if the user has been recently visiting sports sites, it may be a good idea to pop up jersey ads for him/her. Our goal here is to analyze the accuracy with which web page classification (for the genre-based labeling scheme) can help build a useful browsing profile for the user browsing sessions. We define a browsing session as a sequence ofNweb page downloads by a single user. We synthetically generate browsing sessions by randomly selecting a sequence of N web pages using a Markov model of web browsing — we obtain a list of the transition probabilities of the top 5 web sites that a user is most likely to visit while on a given web site from Alexa [Chierichetti et al., 2012, Inc.]. We use these transition probabilities and a simple Markov model to generate browsing sessions that includeNweb page downloads by doing the following steps:

TABLE4.9: Accuracy in Identifying Most-visited Genres. Labeling scheme N=20 N=50 N=200 Genre-based labels K=1 86.1% 85.1% 89.2% K=2 89.0% 89.2% 89.2% K=3 83.2% 87.1% 87.1% K=4 84.9% 83.0% 81.8% K=5 80.5% 82.2% 91.8%

1. Select a single browser to download all web pages for a single browsing session.

2. Randomly select and browse a “seed” web page from our list of 3345 web pages using the selected browser.

3. Given the previous web page, select a web site to browse according to the transition probabilities ob- tained from Alexa — if the web site selected is not included in our dataset of 250 web sites, randomly select a web site.

4. Randomly browse a web page in our dataset that is hosted by the selected web site. 5. Repeat steps 3 and 4 untilNweb pages have been browsed.

We perform the above steps to generate 1000 browsing sessions forN = 20, N = 50, andN = 200 — that is, 3000 browsing sessions total. Please note that the steps above do not explicitly download web pages. We are simply grouping previously downloaded web pages into browsing sessions with N web pages — thus, we do not consider the scenario of overlapping traffic which requires using a web page segmentation approach.

We collect statistics on the top-K(in terms of frequency) AGL genres observed in each browsing session — we collect two sets of statistics, one based on ground-truth genre labels, and the other based on classified genre labels.

In Table 4.9, we list the percent of users for which the unordered set of top-Kgenres based on classified- labels matches with the ground-truth labels. We find that the top-5 genres that a user is interested in can be estimated for more than 80% of the users for each of the parameters tested (KandN). The impact of these parameters on the performance of the user-profiles is discussed below:

• We find that the impact ofKon the performance of the user profiles depends on two cases: 1)K≤2; and 2)K>2.

– When K ≤2, the performance for the profiles usually increases as K increases. This result occurs because there are browsing streams where the two most frequently observed classes were classified in the wrong order — that is, the most frequently observed label was classified second and the second most frequently observed label was classified first. In such scenarios, the profile for K=1 would be incorrect whereas it would be correct for when K=2 — recall that our metric for topKlabels preferences is based on unordered sets.

– WhenK>2, the performance of the user profiles usually decreases asKincreases — we believe that this result is reasonable because adding more labels to a profile will increase the chances for errors.14

• We find that increasing N from 20 to 200 usually improves the performance of the user profiles anywhere from 0-10%. We believe that this modest performance improvement for increasing N is due to using more samples of web pages to build more reliable user profiles — indeed, additional samples provide more chances for the web page classification method to correctly classify web pages in a browsing stream. While we observe a modest performance decline of 3.1% when K=4, we believe that this decline is an anomaly (essentially noise).

Overall, these results are highly encouraging and suggest that targeted ad delivery can significantly benefit from web page classification based on just anonymized TCP/IP headers.15

Another piece of useful information that may be needed from a user browsing profile is thefractionof web pages visited by a user that correspond to a particular genre. For instance, if a user visits the top genre for 95% of the web pages that they browse, he/she is unlikely to be interested in ads related to any of the other top genres. We collect statistics on the fraction of web pages that a user visits for each of their respective top- 5 genres (both based on ground-truth labels as well as classified labels). Figure 4.10 plots the median and 95% confidence intervals of these per-user fractions, for their top-5 genre and navigation preferences. We find that the top genre-based and navigation-based browsing frequencies yielded by classified labels align 14_{Though, the performance will increase asKapproaches the number of possible classes.}

15_{It is important to note that even though the classification micro F-score for the AGL class were only around .75, the user browsing}

profiles being constructed here are simply relying on a comparison of thesetsof top-Kgenres (and not correct classification of each of theNweb-page visited).

0 5 10 15 20 25 30 35 40 45 50 Percentage Profile Genre 1 Genre 2 Genre 3 Genre 4 Genre 5 Navigation 1 Navigation 2 Navigation 3 GT ML

Figure 4.10: Frequency of Labels (Ground Truth and KNN classified).

extremely well with those based on ground-truth labels for both genre-based and navigation-based labels. We conclude that web page classification is fairly well suited for building frequency-based user browsing profiles, even for content-genre based labels.16