Integration in the Context of LiveLabs

The system operates in an environment that has been equipped with infrastructure-based mobility tracking technology of visitors entering and navigating within the space. Viewer mobility data is captured using Wi-Fi fingerprinting technology developed and deployed as part of LiveLabs [Jay+16]. Whilst a client device (e.g. a smartphone or tablet) is still required, users are not required to install a dedicated client application such as Tacita. By connecting to a Wi-Fi network available in the space, the location tracking provided by LiveLabs is initialised. For each device connected to the network, backend components provided by LiveLabs compute the location of the client device based on Wi-Fi fingerprinting printing. This is a contrast to the approach taken by Tacita in which the location (and display proximity) is computed by the client device itself by detecting Bluetooth Low Energy beacons. In order to conduct the trials described in the subsequent sections, we implemented the following middleware components:

1. locationpattern recognitionof viewers,

2. capturing a log of real-time viewer location information (context data fetching), and 3. detecting viewer proximity to displays and dynamically delivering the best-matching

content to the viewer (infrastructure connector).

An overview of the data flow including the main components is provided in Figure6.14. Whilst the infrastructure-based personalisation system consists of a total of six components

6.4 Infrastructure-based Tracking 151

initially introduced in Section3.4.2(p. 73), we specifically focus on the components that implement required interfaces to support personalisation.

Context Data Fetching

The Context Data Fetching component accesses external information and makes these available to the remainder of the system. In particular, the following set of contextual data is accessed and processed:

Real-Time Location Tracking Visitors to the experimental space are tracked using Wi- Fi fingerprinting. In order to be tracked by the system, viewers were required to connect to a Wi-Fi network accessible by LiveLabs. Whilst the location information is collected and computed on the infrastructure-/server-side, viewers’ mobile devices serve as beacons as their location is determined based on the Wi-Fi connection and signal radio signal strengths. Accessing the current location information from the infrastructure-servers in real time is a crucial part of the overall system. The real-time location tracking module accesses the location tracking interface of the infrastructure in a fixed time interval of 5 seconds and fetches the current location data of all individuals. This data is provided in the form of a single comma separated values file (each line represents the current location of a user and consists of their anonymised MAC address and location point). The raw data is processed and parsed, and subsequently stored in the contextual store. The location data further serves as a foundation for subsequent processing such as pattern recognition.

Display Configuration Similar to other public display systems, we had to provide a mechanism in which display locations, trigger zones and content schedules could be defined and considered by the system. We therefore designed a simple user interface in which space owners and administrators can define required parameters including trigger zones and the default set of content items that each display is showing. The Context Data Fetcher component retrieves this set of configuration parameters in a regular interval and makes it available to the remainder of the system including the content scheduling and user to display proximity detection components.

After successfully fetching, parsing and storing new location and other contextual data, the processing module additionally distributes a ‘location update’ event to the main event loop, triggering the content selection and proximity detection modules. We note that additional contextual data can be processed and stored by simply creating additional modules within this component.

Pattern Recognition

The Pattern Recognition component accesses historical mobility traces stored within the internal contextual data store in a fixed time interval and performs a set of mobility pattern

6.4 Infrastructure-based Tracking 152

recognition algorithms. The mobility traces serve as a key source for gaining an understanding of viewer preferences but are not linked to individuals using their clear names or other identifi- able information. Instead, mobility traces are linked to hashed MAC addresses providing a basic level of anonymity whilst still enabling the recognition of individuals returning to the space. Locations of people present in the space are captured in approximately 5-20 second intervals (depending on the processing load on the external Wi-Fi tracking system) and stored in the internal Context Data Store.

The system was designed modularly and supports the execution of multiple Mobility Pattern Recognition algorithms on the same dataset. To demonstrate the levels of insight that can be gained from analysing the location traces, we designed a simplelocation scoring

algorithm. The algorithm requests mobility traces of the previous day, and maps all location points to rooms. In a subsequent step, it computes thetotal time spentper user and room. As a result, the algorithm provides a scored list of the top five rooms in which users have spent the most time throughout an entire day—and discards the remainder of the places a user visited. This result is written back into the Context Data Store and made available to other components of the system.

Infrastructure Connector

In order to support the content selection and scheduling, we developed a novelspatio-temporal

content schedulingapproach. The decision which content to show at a certain location and

point of time is highly dependent on its current context, particularly the viewers present in the proximity of the display and the analytical insights gained both about the viewers. Therefore, we define the decision on which content to show as a function dependent on the viewers identity (used to retrieve their preferences), their location and the current date and time. Equally, space owners and content providers need to be able to create display schedules that define under which circumstances content is delivered to displays and viewers.

Through the definition of ‘rules’, we allow the specification of the content that is to be shown on displays based on three parameters: the location in which someone has spent the most time (due to the use of the simple scoring algorithm), the date and time frame for which the score was computed, and the content that is to be delivered out. We support the definition of an arbitrary amount of rules, allowing the creation of complex content schedules and supporting the consideration for viewer interests.

During a content scheduling process, the content selection components first accesses the current set of location points of anyone present in the space (Figure6.14). In a subsequent step, the module iterates over every person’s location and compares it with the predefined trigger zones of displays to detect whether an individual has been in proximity to a display. If an individual was detected in proximity of a display, the system accesses the contextual data store to find matching rules based on the above defined function (i.e. the viewers anonymised identity, the location and the current date and time). If a single matching rule was found, the back-end selects the associated piece of content to be delivered out to the display that belongs

6.4 Infrastructure-based Tracking 153

Figure 6.15:Flow diagram for infrastructure-based personalisation (display interfaces).

to the matching trigger zone. If multiple rules have been associated to an individual’s context, or multiple viewers have been detected in the proximity of a single display, appropriate conflict resolution mechanisms need to be performed to determine the final piece of content to be shown on the display. A simple solution, for example, can be the selection of a rule and content at random but further, more complex algorithms for such conflict resolution can be developed.

Due to the content selection on the infrastructure-side, we a simple Web-based client is required on the display side in order to visualise the selected content items (Figure6.15). The display client consists of an communication interface to retrieve new content scheduling commands from the infrastructure. After receiving a content scheduling request from the back-end server, the request gets passed on to the Content Parser module for parsing and verification. Content scheduling commands consist of a simple description and hyperlink to the content (in the form of a simplified version of the Content Descriptor Set) which gets passed on to the Content Visualiser module. This module retrieves the content from the remote location and shows it on the display until the arrival of the subsequent content scheduling command.

Implementation

The display client (content visualisation component for digital signs) was implemented on top of AngularJS, a JavaScript framework, and HTML. The back-end components and infrastruc- ture connectors were implemented on top of the Tornado Web Framework and Python 3. Each component of the back-end system are implemented as individual Python-based modules, whilst content data fetchers (real-time location tracking and display configuration) are repre- sented by Python classes, enabling a simple extension with additional context data fetching modules. Infrastructure connectors are, likewise, implemented as individual modules provid- ing the ability to support a set of heterogeneous display infrastructures using a single back-end instance.

6.4 Infrastructure-based Tracking 154

6.4.1.2 Trial Context and Collected Datasets

To support the evaluation of our infrastructure-based tracking mechanism, we conducted a controlled experiment in a real-world setting in the context of a large convention centre in Singapore. The convention centre spans across approximately 42,000sqm, over six stories and can hold up to 10,000 visitors. As previously described in Section3.4.2, we utilise the LiveLabs [Jay+16] infrastructure deployed at the convention centre to retrieve current location readings. Due to the high visibility of the digital signage deployment, and limitations in the deployment of prototypical research system in the context of a commercial space, we conducted a set of controlled walk-by experiments only. Walk-by experiments allowed us to gain insights into the performance of the overall system, and the suitability of Wi-Fi-based location tracking for display analytics purposes and for the delivery of personalised content. For designing and conducting the walk-by experiments, we followed a methodology initially described by Davies et al. [Dav+14] in which they introduced two key metrics:

content accuracyandcontent exposure[Dav+14] defined as follows:

Content Exposure defines the proportion of time a personalised piece of content is visible on the screen while the viewer is in the viewable area of the display, i.e. the exposure of the content to the viewer. [Dav+14] describe this metric as “the effectiveness of the system at showing content to the viewer.” From a display analytics perspective, content exposure is a direct result of the timely and accurate detection of a viewer in the visible area of a display and therefore fundamental to the computation of accurate analytics insights.

Content Accuracy defines the proportion of time the requested (personalised) content could have been seen by the viewer whilst the viewer was present in the viewable area of the display. Whilst this metric appears to be less relevant from a display analytics perspective as it is purely focused on content delivery aspects, capturing the content accuracy provides insights into the performance of the overall system and into the accuracy of the location tracking technique used.

We designed an experiment around the content exposure and content accuracy metrics by conducting a series of controlled walk-bys. We identified five representative display locations and walking routes within the convention centre (Figure6.16):

Central A typical display located on the main floor of the conference venue with the viewer walking towards the display whilst remaining on the same floor level throughout the experiment.

End of corridor (1) The display is located within the main floor of the conference venue on the edge of the area that is covered by the Wi-Fi location tracking system. The viewer is walking towards the display and turning around the corner at the end of the path.

6.4 Infrastructure-based Tracking 155

Meeting rooms

Meeting rooms Meeting rooms

Meeting

rooms Meeting rooms Meeting rooms (a)Central Meeting rooms Meeting rooms Meeting rooms Meeting

rooms Meeting _rooms Meeting rooms (b)End of corridor (1) Meeting rooms Meeting rooms Meeting rooms Meeting

rooms Meeting rooms Meeting rooms (c)End of corridor (2) Meeting rooms Meeting rooms Meeting rooms Meeting

rooms Meeting rooms Meeting rooms (d)Side entrance Meeting rooms Meeting rooms Meeting rooms Meeting

rooms Meeting rooms Meeting

rooms

(e)Main entrance

Figure 6.16:Floor plans showing the main floor of the convention space for each of the five exper- imental settings visualising the display location and its orientation (green circle), differently sized trigger zones (small, medium and large), and the paths of the walk-by experiments. Dots represent an approximation of the granularity of location points provided by the Wi-Fi location system (the floor plans provide an approximation only and have not been drawn to scale).

6.4 Infrastructure-based Tracking 156

End of corridor (2) This display location is similar to ‘end of corridor 1’ but is located at the opposite end of the conference venue.

Side Entrance Whilst the previously mentioned display locations focused on the viewer remaining on the same level throughout the experiment, we further investigated the potential interference of the viewer changing floor levels throughout the experiment. This display is located on the main floor of the convention centre, however, the viewer appears in the trigger zone of the display (and enters the boundaries of the location tracking coverage) through an escalator from the level below.

Main Entrance To further investigate the potential interference of the viewer changing levels, we chose the displays at the main entrance as the final location. In contrast to ‘side entrance escalator’, the viewer transitions across three levels of which the bottom level and the main floor are covered by the location tracking system. The display is located on the main floor just at the end of the escalators.

In addition to different display locations and their characteristics, we varied the trigger zone (i.e. the spatial boundaries which viewers have to cross to be recognised in the viewable area of the display and trigger a content request) from small (i.e. trigger zone is identical to the viewable area of the display), medium (i.e. trigger zone approximately doubled in size) and large (i.e. trigger zone additionally increased and optimised for a high content exposure). For each display location and trigger zone size, we conducted ten repeating walk-bys, capturing the following timestamps: viewer entering the viewable area of the display, viewer leaving the viewable area of the display, content appearing on the display (i.e. the time the system recognised the viewer in the viewable area of the display) and content reverting back to the normal content. In the subsequent section, we provide detailed insights into the accuracy and performance of the system by focussing on content exposure and content accuracy metrics.