Display Personalisation Retention Analytics

The dataset captured through Tacita can be used to create an additional set of analytics reports specific to describing viewer engagement and interactions with Tacita. In contrast to analytics reports captured for purely commercial- and advertisement-based public displays and billboards, the requirements for analytics reports in the context of Tacita are around revealing and visualising insights into the viewer behaviour and usage of personalisable display applications such as the ways in which users configure content and the spatial distributions of requests for personalised content.

4.2 Analytics Based on Viewer Data 99

Figure 4.8: Growth of Tacita users through- out the deployment (blue) and unique num- ber of users per day (red) (initially published in [Mik+18d]).

0

10

20

x

0.00

0.25

0.50

0.75

P(

X>

x)

Figure 4.9:Cumulative distribution function of frequency of revisiting the configuration pages of individual Trusted Content Providers (ini- tially published in [Mik+18d]).

4.2.5.1 Usage and Interactions

The assignment of a globally unique identifier to requests originating from Tacita users enabled us to recognise recurring users and created viewer-centric usage reports throughout the lifetime of the deployment. This included insights into the usage and interactions with Tacita including the reporting ofunique users(Figure4.8).

Tacita is characterised by a set of unique interaction patterns that differ to these found across mobile phone applications and traditional, interactive public displays. For example, unlike interactions with many mobile phone applications, Tacita users are only required to activate and configure the system once on their mobile devices. While passing by displays that support display personalisation, users implicitly interact with displays and applications simply by their phones detecting proximate displays and requesting applications that users have previously activated – users are not required to actively launch Tacita on their mobile device for further use.

Considering these unique interaction characteristics, we have created a set of reports that allow display owners and content providers to better understand the usage patterns and explicit interactions with configuration pages. Besides standard Web analytics that can be captured on the Web-based configuration pages, we considered each time users accessed and changed their preferences and calculated a cumulative distribution function (Figure4.9). The figure visualises the proportion of users who revisit the configuration pages a number of times – expressing, in this case, that the majority of Tacita users configure their applications once and only a very small proportion of users revisit the page again to adjust configuration parameters. Such insights into interaction patterns can be crucial for the design of personalisable applications. For example, low frequencies of revisiting configuration pages may emphasise the importance of creating a good user experience as users are likely not to revisit configuration pages. We note that due to the nature of the configuration pages, user interactions within these pages, e.g. button clicks, scrolling behaviour and configuration parameters, can be captured and reported using standard Web analytics techniques.

4.2 Analytics Based on Viewer Data 100

0 10 20 30 40 50

Days since first use

0.0 0.2 0.4 0.6 0.8 1.0 Proportion of users

(a)User retention report based on display personalisa- tion requests .

0 10 20 30 40 50

Days since first use

0.0 0.2 0.4 0.6 0.8 1.0 Proportion of users

(b)User retention report based on configuration changes of personalisable applications.

Figure 4.10:Tacita user retention reports with aper-daygranularity (initially published in [Mik+18d]).

0 1 2 3 4

Weeks since first use

0.0 0.2 0.4 0.6 0.8 1.0 Proportion of users

(a)User retention during winter term.

0 1 2 3 4

Weeks since first use

0.0 0.2 0.4 0.6 0.8 1.0 Proportion of users

(b)User retention during spring term. Figure 4.11: Tacita user retention reports with a per-week granularity (initially published in [Mik+18d]).

4.2.5.2 Retention Rates

A common way to report on the loyalty of customers in retail or users in a mobile phone setting

areretention rates[RZ93]. Such retention rates can be used to describe the success of mobile

phone applications: if users are still accessing a mobile phone application after multiple days, weeks or months it is considered as an indication for a successful application [Pel+18]. In the context of personalisable public displays, similar reports can be created to describe the lifespan of users requesting personalisable content on the display network. Whilst in mobile phone usage statistics retention rates are computed by considering the time span between the first and last time a user has opened an application, the metric can be adapted to our use case by considering the first and last time viewers requested personalised content on displays or explicitly visited the configuration page of a personalisable application.

To demonstrate the potential of the insights that can be gained from such retention reports, we explored two different types of reports with different granularities. Firstly, we consider retention reports on a per-day granularity showing the proportions of the activity lifespans

4.2 Analytics Based on Viewer Data 101

based on the days counted between the first and last captured requests of viewers and between the first and last interaction with application configuration pages (Figure4.10). Secondly, due to the nature of display personalisation systems (i.e. viewers are required to pass by supported displays in order to be considered in the reports), we additionally created reports with a lower granularity on a per-week basis accounting for the fact that some viewers could be still using the system but happen to not pass by a supported display (Figure4.11). Figure4.10aindicates a relatively high retention of continuous personalisation requests: after ten days, over 50% of users continued issuing personalisation requests implicitly by walking by displays indicating that Tacita has not been removed or disabled. In line with the cumulative distribution function of frequencies of revisiting configuration pages shown in Figure4.9, the retention rate for configuration changes shown in Figure4.10bshows a low retention rate after the first day indicating that users very rarely revisit configuration pages to change their preferences. Considering retention figures on a finer time granularity (Figure4.11), we observe that over 50% of viewers are still successfully requesting personalised content one week after beginning to use the system – a metric that can be observed both during winter and spring terms (Figures4.11aand4.11brespectively). We further observe that whilst the majority of viewers cannot be observed after two weeks of usage, the proportion of ‘long-term’ users remains stable across weeks 2, 3 and 4.

4.2.5.3 Stakeholder Analysis

The reports outlined above provide the following set of benefits to individual stakeholder groups.

Display Owner Display owners can utilise retention rates to determine the perceived utility of the content (or personalisable applications) offered to the viewers. Very low retention rates of certain categories of applications may suggest that the overall offering of personalisable applications (and content) can be improved.

Space Owner Space owners gain insights into the popularity and uptake of novel tech- nology deployed in their space. In the context of Tacita, for example, space owners would likely have been involved in the installation, deployment and advertisement of the application to visitors of the space. The use of retention rates can help space owners understand their ‘return on investment’ and potentially have an influence on the future deployment of technology for improving the experience of visitors or customers of the space.

Content Provider Retention rates in the context of digital signs can be used to inform the design of personalisable display applications and the content offered. For example, we noted very low retention rates of users accessing configuration pages of personalisable applications suggesting that users of such a system only configure the application once or very few times. Such usage patterns may introduce a set of system design requirements that may impact the design and development of applications.