5.4 Distributed Inference for HDP on Apache Spark
5.5.2 Robust Pervasive Context Discovery
In this section, we demonstrate our proposed framework for context acquisition. We use two popular large datasets Student Life (Wang et al., 2014) and Mobile Data Challenge (Laurila et al., 2012) for analysing pervasive signals with applications in pervasive and human dynamics application.
5.5.2.1 Datasets and Experimental Settings
The StudentLife and Mobile Data Challenge (MDC) large and real-world datasets that are frequently used in pervasive and ubiquitous computing research. The former is collected from smartphones of 48 students at Dartmouth College over a 10-week spring term in 2013, while the latter are collected from smartphones of 191 parti- cipants in 18 months from 2009 to 2011 at Lausanne, Switzerland. Both datasets contain multi-channel data (e.g., SMS and call history, accelerometer, etc.). How- ever, in our experiments, we limit their use only in Bluetooth and WiFi signals. We applied similar procedure mentioned in (Nguyen et al., 2016a,b) to pre-process the two datasets. That is, for WiFi data, we eliminate access points that have been scanned less than 100 times and for Bluetooth data, we eliminate Bluetooth devices that have been scanned less than ten times as they are not statistically meaningful and would not affect the topic discovery. The only different point is that in this paper, a Bluetooth and a WiFi scan are considered to be from the same scan if their timestamp difference is less than 5 seconds, instead of having the same timestamp as in the two aforementioned papers.
In the first experiment, we compare our inference framework with Gibbs sampler in (Nguyen et al., 2016a,b) on two datasets. We would like to demonstrate the advancement regarding of running time of our proposed method but still obtaining comparable performance.
In the next experiment, we propose a new representation of data for learning HDP with product space in our proposed framework. Our observations in StudentLife dataset include two channels: WiFi and Bluetooth. The dimensions for channel
5.5. Experiments 114
observations are x and y respectively. We merge these two channels into one using a dimension of x×y. After learning, we can marginalise this joint distribution to obtain the atoms for each channel. We then compare our results with Gibbs sampler in (Nguyen et al., 2016b) which is available for StudentLife dataset only. We then proceed the qualitative analysis for MDC datasets.
5.5.2.2 Learned Patterns from Pervasive Signals
We compare our proposed framework with existing works which applied HDP with Gibbs sampler (HDP-Gibbs). Table5.4presents clustering performance comparison between our method (SkHDP-CVB) with HDP-Gibbs on StudentLife dataset. We also compare the running time of two methods. In comparison with Gibbs sampler, our method outperforms on RI and are still comparable on other metrics. However, the time for running our method for 100 epochs is significant (36 times) less than required time to run Gibbs sampler for 500 iterations.
We visualise each topic by 3 tag-clouds. Firstly, as the model produces a number of topics where each topic is a distribution over WiFi hotspot IDs, we can visualise each topic by a tag-cloud of WiFi hotspot IDs. The size of each WiFi ID is proportional to its contribution to the topic. Moreover, the MDC dataset also provides a table visits_20_minutes, which contains the places and the time that participants have been visited for at least 20 minutes. By matching the time of WiFi scans and the time from the table visits_20_minutes, we can infer a place ID for each WiFi ID. Thus, we produce another tag-cloud in which WiFi IDs are replaced by theirs corresponding place IDs. Furthermore, the dataset also provides a ground-truth table places in which users give a label for each place ID. Hence, we associate the inferred place IDs and the participant ID, which corresponding to the phone where the scans took place, to produce labels (e.g. 5980_home is the home of participant 5980).
Visualising topics as 3-tuple tag-clouds can help us infer some meaningful contexts of the topics. An example is the visualisation of topic 7 (Fig. 5.6). From this visualisation, we can see that this topic is a group of WiFi hotspots located around location number 1, 2, 3, 4 and 5, which are labelled as the home of many users. Thus, we can infer that topic 7 is highly likely a residential location or adormitory.
To facilitate the exploration of relationships between location/proximity topics and WiFi/Bluetooth IDs, we employ the GEXF Viewer tool and display discovered topics as a graph. Fig. 5.7shows the network built from discovered topics, where nodes are composing from 4 different classes: WiFi ID (green), location topic (red), proximity
5.5. Experiments 115 L1 L2 L3 L4 L5 L6 L7 L8 L9 L10 L11 L12 L13 L14 L15 L16 L17 L18 L19 L20 71 79 81 83 94 95 96 118 119 121 130 131 229 253 271 280 323 348 353 423 496 580 581 583 593 620 622 623 657 658 667 702 717 718 767 771 773 839 841 908 973 1068 1200 P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12 P13 P14 P15 P16 P17 P18 P19 P20 1 4 5 6 11 12 15 19 21 22 23 24 29 34 36 43 44 48 49 51 54 56 57
Figure 5.7: The relationship of discovered topics and Bluetooth/WiFi IDs with StudentLife data. Nodes and edges are: (1) WiFi ID; (2) Bluetooth ID; (3) The edge means that the Bluetooth ID 52 contributes the most to the proximity topic 9; (4) Proximity topic; (5) This edge links a proximity topic with its corresponding location topic; (6) Location topic.
topics (aqua) and Bluetooth ID (purple). The size of each location/proximity node is proportional to its contribution in the location/proximity topic mixture. For each topic, we visualise the connection with the 5 IDs that contribute the most to this topic. For example, when we choose the proximity topic 9 (P9), the viewer will highlight the 5 Bluetooth ID nodes 21, 24, 34, 57, 44 to indicate that these are the 5 most contributing IDs to the proximity topic 9. Moreover, the connection edges size also relatively reflects the contribution of each node to the topic. We also use GEXF Viewer tool to interactively display discovered clusters and data as a graph.
5.6. Conclusion 116
5.6
Conclusion
We have presented a scalable method for inference one of the most important class of model in Bayesian nonparametric clustering – the Hierarchical Dirichlet processes. The derivation is obtained only for coordinate descent variational inference. How- ever, it is straightforward to use the results for stochastic variational inference (Hoff-
manet al.,2013). Another direction of the extension includes applying the methods
to more advanced models which use the same hierarchical modelling mechanism as in HDP such as nest Dirichlet process (Rodriguezet al.,2008), our context-sensitive DP developed in Chapter 3 and our subsequent Bayesian nonparametric multilevel clustering models developed later on in this thesis. The experimental results demon- strate that our methods are faster than the existing variational inference methods while yielding the better model quality. Most importantly, our work enables to scale seamlessly with data and the applicability of HDP to modern real-world datasets which can contain millions of documents.
Chapter
6
Scalable Bayesian Nonparametric Multilevel
Clustering
S
cience, engineering, and technology in the era of big data are contributing to the explosive growth of data with highvelocity. A consistent research theme of this dissertation is to develop richer, fast and sophisticated models through the theory of Bayesian nonparametrics, graphical models and approximate inference. Besides massive data stream, big data also has large (petabyte and exabyte) scales, wide variety and considerable uncertainty. For instance, the textual data from Wikipedia are growing vastly in spectra day by day, while there are other side information and metadata associated with each article such as the article editors, topic categories, authorship information, time-stamp, publication venues and so on. For scientific publication, PubMed maintains a growing collection of medical related publications around the world. Apart from the abstract, each paper also includes valuable meta- data such as authors, titles, and medical heading subjects. In this chapter, we will tackle clustering problems in such massive data stream containing variety data (and meta-data) fields.A prominent feature in numerous modern datasets tackled in machine learning is how the data are naturally layered into groups in a hierarchical representation: text corpus as collection of documents, which are subdivided into words; user’s activities are organized by users, whose sessions divided into actions; electronic medical records (EMR) organized as sets of ICD1 codes diagnosed for the patients.
Probabilistic modelling techniques for grouped data have become a standard tool in machine learning, including topic modelling (Bleiet al., 2003;Teh et al.,2006) and multilevel data analysis (Hox, 2010; Diez-Roux, 2000). Another important feature in such datasets is the availability of rich sources of additional information known as contextsandgroup-specificmeta-data. These include information about authorships, timestamps, various tags associated with texts and images, user’s demographics, etc. For consistency, we shall refer to the content groups (e.g., text documents, images,
1Stands for International Classification of Disease.