Context Aware Personalized Ad Insertion in an
Interactive TV Environment
Amit Thawani, Srividya Gopalan and Sridhar V.
Applied Research Group Satyam Computer Services Limited
#14, Langford Avenue, Lal Bagh Road, Bangalore, India – 560025
Abstract. The growth of digital television over satellite, cable as well as terres-trial networks has driven advertisers to ‘better target” users by local personal-ized advertisements, which generates qualified lead for future sales. Context awareness in ITV applications can be used as a powerful tool to deliver most relevant and personalized ad based on user’s context. In this paper we have proposed architecture for context aware real time selection and insertion of ad-vertisements into the live broadcast stream by taking into consideration the user’s current and past contextual information. Our approach is based on aggre-gating a past sequence of individual contexts and associating the current user activity to those past contexts to determine the best ad to be delivered relevant to the current activity.
Introduction
The growth of digital television over satellite, cable as well as terrestrial networks has driven advertisers to ‘better target” users by local personalized advertisements, which generates qualified lead for future sales. The advent of Personal Video Record-ers (PVRs) and Video on Demand (VOD) is viewed as a threat by advertisRecord-ers as they create ad-skippers. As the viewers started evading advertisements, ad agencies com-pete on innovative ways to ensure their ads are viewed. There is a strong need to re-work on TV advertising to create innovative features that can exploit advertisement opportunities. Personalization in interactive and future televisions aims in targeting content to individual users by adapting the content based on users’ likes and circum-stances. [9] compares different types of addressable advertising and demonstrate su-periority of home addressable advertising. [4] presents a life-style based approach for the delivery of personalized advertisements based on segmentation variation and simi-larity variation. In a related paper [7], we have proposed a Channel Surfing Analysis (CSA) algorithm for predicting the “user(s) in front” of TV by a dynamic analysis of channel viewing characteristics. Such user identification can be used along with a Home Information System (HIS) to categorically decide about the nature of ads that are likely to have an impact.
Context-aware and situation-aware services together enhance the perceived quality of the delivered information. Context awareness provides mechanisms for developing
adaptive applications, which is an important aspect of ITV applications. [3] presents a context-aware information retrieval system that analyzes the user’ s history to derive an enhanced context. [5] proposes to model the user actions from contextual informa-tion, in a way to ease its interaction with context-aware ITV applications. Advertisers create multiple versions of advertisements for a single TV spot in order to cater for different user communities. Given multiple versions of advertisements, and also in-formation about user currently in front of the television, the challenge is to make use of and analyze the user’ s current and past contextual information to select and insert the most appropriate advertisement from the multiple versions of multiple advertise-ments at real time.
Though research has been done for personalized ad insertion that considers users preferences and user profile information, the use of complete contextual information for personalization needs to be addressed. In this paper we have proposed an architec-ture for context aware real time selection and insertion of advertisements into the live broadcast stream by taking in to consideration the user’ s current and past contextual information. Our approach is based on aggregating past sequence of individual con-texts i.e. past viewing patterns and associating the current user activity to those past contexts to determine the most appropriate ad to be delivered relevant to the current activity. The proposed architecture is realized by means of four distinct subsystems, CoD (Context Derivation), CaASI (Context-aware Ad Selection and Insertion), UId (User Identification), and BAR (Bulk Ad Retrieval).
Context
Context can be described as information that can be used for characterizing the status of an entity in one specific case. One entity can be a one person, one place or one object relevant for any type of interaction between user and application, including the user and the application itself [1]. Parameters such as: a) where is the user; b) who is the user; c) how the user works; d) when the activity is being done; and e) what the user is doing; are used to build a context aware environment. [1] defines the four enti-ties viz. location, entity, activity and time to address the parameters for building a context aware environment. [6] discusses which type of contextual information could be used in an interactive environment.
Fig. 1 describes the general context information comprising of location, identity, activity and time and more specifically context information for ITV applications. Lo-cation information of the user at home such as “user is in living room” or “user is in bedroom” can be used to determine the location context of the user. Identity informa-tion in ITV environment for context can be characterized as user identity, device iden-tity, content identity and event identity. User identity is specified by user profile as described in the HIS stored in the Set Top Box (STB) at home. User identity typically includes information such as user’ s name, age, occupation, gender, marital status, fa-vorite channels, movie interests, music interests, ad interest, etc. Device identity specifies attributes such as resolution, features, connectivity, etc. of multimedia de-vices at home. Content identity is stored in the STB as Electronic Program Guides (EPG) after being retrieved from the broadcast. The EPG also consists of other
pro-gram related information such as category, channel type, genre, parental guidance, summary, etc. The Event identity is determined either by the broadcast or by other sources such as sensors and the network. Event information can be for national or lo-cal events that might have an impact on the user context or user’ s TV watching be-havior.
Fig. 1: Context Information
Morning Weekend Weekday Evening Device Activity Past Activity Pattern Group Activity User Activity Device Identity User Identity Event Identity Content Identity Presence in Living Room Presence in Entertainm ent Room Presence in Kitchen Context
Location Identity Activity Time
Activity information for context can be characterized for user activity, device ac-tivity and user/device history. User acac-tivity such as “ user watching TV with family”, “user watching TV while working from home” , “user watching TV while doing house-hold jobs” , etc. can be used to derive the user’ s context information for the activity entity. Device activity information typically includes the device usage information by the user. User and device history information typically includes users interaction with interactive controls of programs and ads in the past, his past purchases, websites and web pages visited in the past and channels and programs viewed in past. Time infor-mation for context consists of the current time as well as time inforinfor-mation of the users history for channels/programs, purchases and interactivity. Time information gives cues for the context of the user as user’ s TV watching behavior changes for different time of the day as well as different days of the week.
Our proposed approach considers context information as Current context (CCo) and Analyzed or AggregatedContext (ACo). CCo is defined by the current activity and the entities associated with it. CCo is the information available at hand at an in-stance of time from direct sources. In our proposed system past context sequences are analyzed and aggregated to derive usage trends and user preferences, in the form of rules which are defined as ACo. For the ITV application of personalized ad insertion, based on CCo and ACo classification discussed above, the CCo information contains identified user watching TV at that instance of time, the user’ s age, the current gram schedule for all the channels derived from the EPG, the program type, the pro-gram category, the propro-gram genre and parental guidance of the propro-grams derived from EPG, ad slots in the programs and also ad sponsors. Similarly, the ACo informa-tion would contain derived user preferences for the programs, program types, program category and program genre, user’ s demographic information, user’ s viewing behav-ior, user’ s remote usage pattern, user’ s viewing history, user’ s purchase history, user’ s interactivity history and local and national event information.
The XML-based MPEG-7 (also called Multimedia Content Description Interface) Description Definition Language (DDL) [2] along with TV-Anytime Metadata Speci-fication [8] can be used as a representation format for metadata to represent and
de-scribe components of a broadcast, ads and user profiles in the HIS. The representation format of the content enables determination of keywords describing the content. Simi-larly, keywords can be determined for user attribute information as described in the HIS. Since the context information is based on the content and user attribute informa-tion, a set of keywords can be determined to represent the context information. These determined context information keywords are matched against the keywords repre-senting the ad content. The ads whose content description keywords have a match with the context information keywords are identified as the targeted ads to be inserted and presented to the user.
System Architecture
Multiple distinct networks are interconnected to deliver broadcast content to home users. The broadcast network broadcasts content to MSOs through radio channel and MSOs deliver content to home users via a cable network. From the point of view of ad insertion, a typical approach is to insert the ads before broadcast. This gives little opportunity to target ads to obtain a better return on investment. While content reaches MSO through radio, the related ads could reach the MSO through Internet or an IP network. In this case, the MSO stores these ads and could insert them at an ap-propriate place before forwarding the content to the home users. The third approach is to distribute the relevant ads directly to a home network where a set-top box would do the necessary ad insertion into the content before displaying the same. The last two approaches provides an opportunity to select and insert “ the most appropriate” ads.
The system level description of our proposed approach is depicted in Fig. 3. As de-scribed in Fig. 3., the architecture consists of four distinct subsystems, CoD, CaASI, UId, and BAR Functionality of The CoD subsystem involves determination of both the CCo and the ACo information. The ACo information is used to identify ad tags, which depict analyzed and aggregated context of all users at home, for requesting ads to be retrieved from the MSO. The ACo information, along with inputs from the UId subsystem, is also used to identify appropriate ads from the ad database to populate secondary ad storage (cache).
The UId subsystem implemented on an STB is responsible for processing home users’ viewing characteristics. Information such as user’ s likes and dislikes (such as with respect to movies, sports and music), his/her recent purchases, typical viewing hours on weekdays and weekends, and typical channels viewed are used to analyze the received viewing characteristics to identify one or more users (joint viewing) who might be watching TV [7]. The BAR subsystem is responsible for requesting ads from the MSO based on the context information of the users at home. Considering the number of sponsors and various versions of ads for each product of the sponsor, it is difficult to store all the ads in the STB. This subsystem solves this issue by requesting ads based on analyzed and aggregated context of all the users at home. It receives the context information of all the users at home from the CoD subsystem module and identifies ad tags depicting the context information. It then requests MSO to send ads based on the ad tags thus determined. The CaASI subsystem is responsible for selec-tion and inserselec-tion of ads in the broadcast content, based on the context informaselec-tion,
for presentation to the user. This subsystem determines points of ad insertion in the broadcast stream by decoding digital cue tone messages. Targeted ads are identified using CCo and the ACo information and further inserted in the broadcast stream in the STB.
Fig. 3: System Architecture for Ad Selection and Insertion Home MSO Receive Multiple Channels Multiple Channel Insertion Transmit Channel Contents Bulk Ad selection Content Ads Receive Ads Receive Content Receive Channel Contents Channel
Selection DecodingChannel Channel Display Send Ad Tags Home Info System
Home Home Home
Channel Mixing Ad Cache CC based Ad Selection Channel content insertion Ad DB Bulk Request processor AC based Ad Tag Selection Ad Tags Aggregation Receive Ad Tags Collect Viewing Characteristics Channel Surfing Analysis CaASI UI Context Derivation (CC and AC) BAR History DB
Home Home Home
Scenario
Consider a home with four members consisting of a dad (John) and mom (Cindy) and 2 kids (Andy and Samantha). The User profile information stored in the STB de-picts: John is an insurance agent and is a sports lover; John’ s favorite sport is basket-ball; John generally watches TV from 7 to 10 in the evening; Cindy works for a bank; Andy and Samantha are school going kids. History information stored in the STB de-picts: John had been visiting websites searching for a basketball jersey; John had pur-chased tickets for the finals of the college league basketball; John has been watching ads showing sports jersey and had interactions with the ITV components of type “ con-tact me” for these ads; Cindy had been looking for vacuum cleaners on websites; Andy and Samantha had been interacting with ads for “ back to school” products. The Events database depicts: Final of college league basketball tournament is on the com-ing weekend; Wal-Mart has sale for home equipment on the comcom-ing weekend. The EPG information depicts: Nike, Wal-Mart and Honda are sponsors for live coverage of college league basketball tournament; Nike has scheduled to show ads for its new range of golf shoes; Wal-Mart has scheduled to show ads for its upcoming sale of home equipment.
Consider a working day and the time 8:00 P.M. John comes back home and switches on the TV and starts watching ESPN channel which is showing live cover-age of college league basketball championship. When the TV is switched on, the ACo component analyzes the history data and user profiles for all members of home. From the events database, the ACo component determines that the final of college league
basketball tournament is on the coming weekend. Similarly, the ACo component, based on user profiles and history information, determines that Cindy has been ning to buy a vacuum cleaner from Wal-Mart and Andy and Samantha had been plan-ning to buy school bags from Wal-Mart. Based on the information determined, the ACo component determines rules and passes on the information to the BAR subsys-tem. Typically the rules that can be determined for the scenario under consideration are: Since John purchased tickets of the college league final match, he would like to wear Nike basketball jersey when he goes to watch the final match; Since Cindy had been looking for vacuum cleaners on websites, she would probably be interested in buying one from Wal-Mart sale on home equipment; Andy and Samantha would be interested in buying “ back to school products” from Wal-Mart since they had been in-teracting with such ads.
The BAR subsystem determines keywords such as <sports, jersey, basketball, col-lege league tournament, vacuum cleaner, kids, schooling, school bag, Northface, Home equipment> and requests the MSO for ads matching the context information. The MSO, on receiving the request, sends the following ads to STB at home: Nike ad for recently launched golf shoes; Nike ad for Nike jersey with basketball theme; Nike ad for soccer shoes with football theme; Wal-Mart ad for the weekend sale on home equipment; Wal-Mart ad for “ always low prices” ; Wal-Mart ad for “ back to school” products; Wal-Mart ad for gardening equipment; Honda ad with F1 theme; Honda ad for new coup model.
Further, the ACo component, based on user profile information, determines that only John and Cindy might be watching TV at this instance of time. Based on the rules determined, it then analyzes the received ads and caches Nike and Wal-Mart ads in secondary ad storage. On remote click, the UId subsystem analyses user profiles and remote usage pattern and determines that the most probable user watching TV at that time should be John. Based on inputs from the UId subsystem and also based on user profile and EPG information, CCo component of the CoD subsystem forms rules which determines the CCo. The CaASI subsystem analyses the CCo information and ads in the secondary ad storage and determines that the Nike ad with the theme “Nike jersey with basketball theme” is the most appropriate ad as compared to the scheduled ad for recently launched golf shoes. The CaASI subsystem then inserts the ad in the broadcast for presentation.
Ads Requested Based on ACo Probable Users at Home
Ads Cached Based on ACo and Context Information History
Identified User at Home
Most Appropriate Based on CCo and Inputs from UId Nike ad for recently launched golf
shoes; Nike ad for Nike jersey with bas-ketball theme; Nike ad for soccer shoes with football theme; Wal-Mart ad for the weekend sale on home equipment; Wal-Mart ad for “ always low prices”; Wal-Mart ad for “ back to school” prod-ucts; Wal-Mart ad for gardening equip-ment; Honda ad with F1 theme; Honda ad for new coup model.
John and
Cindy Nike ad for Nike jer-sey with basketball theme; Wal-Mart ad for the weekend sale on home equipment; WMart ad for “ al-ways low prices”; Wal-Mart ad for “ back to school” products
John Nike ad for Nike jer-sey with basketball theme
Similarly, if the identified user by the UId subsystem would had been Cindy and not John, the most appropriate ad de-termined by CCo component would be:
John and
Cindy Cindy Wal-Mart ad for the weekend sale on home equipment; WMart ad for “ al-ways low prices”
For the same day but at different time, say 5:00 P.M. in the evening, when Andy and Samantha are watching their favor-ite cartoon channel, the cached ads by the ACo component and most appropriate ads determined by CCo component would be:
Andy and
Samantha Wal-Mart ad for “ al-ways low prices”; Wal-Mart ad for “ back to school” products
Andy and
Samantha Wal-Mart ad for “ back to school” products
Summary
In this paper we have proposed an architecture for context aware real time selec-tion and inserselec-tion of advertisements into live broadcast stream by taking in to consideration the user’ s current and past contextual information. XML-based MPEG-7 DDL along with the TV-Anytime Metadata Specification are used as a representation format for metadata to represent and describe components of the system. The proposed work is part of our ongoing work on developing an STB environment with enhanced functionalities and capabilities. The project also involves development of intelligent iTV applications such as user identification, intent tracking, personalized ad insertion etc. We are currently working on a context sensitive cache conscious indexing technique to perform lookups for ad selection.
References
1. A.K. Dey and G.D. Abowd, Towards a better understanding of context and context-awareness, Handheld and Ubiquitous Computing, (H.W. Gellersen, Ed.), Springer, Berlin, pp. 304-307, 1999.
2. ISO/MPEG N3445 MPEG-7, 2001: Multimedia Description Schemes XM (v.7.0). http://mpeg.telecomitalialab.com/working_documents.htm.
3. H. Kammanahalli, Srividya Gopalan, and Sridhar V., Context aware retrieval in web-based collaboration, Proc. Second IEEE Conference on Pervasive Computing and Communica-tions Workshops. Orlando, FL, 2004.
4. G. Lekakos, and G.M. Giaglis, A lifestyle-based approach for delivering personalized adver-tisements in digital interactive television, Journal of Computer-Mediated Communications, 9(2), January 2004.
5. J.B.D. Santos, R Goularte, G.B. Faria, and E.D.S. Moreira, Modeling of user interaction in context-aware interactive television application on distributed environments, Proc. TV01 - Workshop on Personalization in Future TV in Conjunction with User Modeling 2001. Son-thofen, Germany, 2001.
6. J.B.D. Santos, R Goularte, G.B. Faria, and E.D.S. Moreira, The modeling of structured con-text-aware interactive environments, Transactions of the SDPS Journal of Integrated Design and Process Science, v. 5, n. 4, p. 77--93, December de 2001.
7. A. Thawani, Srividya Gopalan and Sridhar V., Viewing characteristics based personalized ad streaming in an interactive TV environment, Proc. 2004 IEEE Consumer Communications and Networking Conference, 2004.
8. TV-Anytime Forum. Specification Series (Metadata). 2003-August. http://www.tv-anytime.org/.
9. V. Dureau, Addressable advertising on digital television, Proc. Second European Conference on Interactive Television, 2004.
