Activity recognition in specific domains

Most of the previous works recognise physical activities (e.g. standing, sitting) or activities in daily lives (e.g. kitchen activities, activities in smart environment). With the prevalence of various sensors and mobile devices, recently many researchers study the recognition of activity in a specific domain. They are not only recognising the target activities, but also monitoring if the activities are performed in a normal way. The most paramount works are as follows:

• Tooth Brushing: In [71], Korpela et al. present a method to evaluate the tooth brushing performing with audio gathered from a smartphone. They first classify different tooth brushing activities with HMMs, and then they use the output (e.g. duration of each tooth brushing activity) of the HMMs to build regression models for the tooth brushing performance scores estimation.

• Locomotion mode: Hemminki et al. [46] present a technique for accurate and fine-grained locomotion mode detection with accelerometer data. They first estimate the gravity component of accelerometer data for calculating the gravity eliminated verti-cal and horizontal acceleration. Then extract multiple features from the acceleration data for building hierarchical classifiers. In [127], the authors propose to use

barome-ter for the detection of transportation mode (e.g. walking, vehicle). The basic idea is that different transportation modes make the pressure change differently.

• Eating: In [168], the authors present a system for nutrition monitoring with a smart table cloth. The table cloth is equipped with a weight sensitive tablet and a fine grained pressure textile matrix, so that they are able to spot different actions based on the pressure change when the users are eating on the table cloth. Thomaz et al. [140]

implement an approach for detecting eating moments with a 3-axis accelerometer on a smartphone. Other works use different sensing modalities such as neck-attached [159]

or ear worn microphones [5] to detect eating related sound for dietary monitoring.

• Smoking: Nguyen et al. [105] present an activity recognition for recognising smoking activity. They try to address the problem of activity recognition in open world where an unlabelled instance can belong to any of the possible activities instead of one of the predefined activities. They propose Multi-class Positive and Unlabelled Learning to reduce the false positive in recognizing smoking in open world. Therefore, they merge the unlabelled instances into the negative set so that the negative set can form a representative set of negative instances, and learning with the positive and negative set can result a correct decision boundary. Kawamoto et al. [63] monitor changes in respiratory rate during sleep with wrist-worn accelerometer, and use the data for the detection of multiple physical conditions such as smoking cessation.

• Sleeping: Hao et al. [43] present iSleep - an individual’s sleep monitoring system using off-the-shelf smartphone. They develop a lightweight Decision-tree-based algorithm to classify the microphone data of the smartphone into multiple sleep related events such as body movement, couch and snore, and use the classification results for eval-uating sleep quality. In [40], the authors move a step ahead and detect sleep stages with sensors on smartphone. The basic idea is that different sleep stages are accom-panied by different body movements and acoustic signals features. They use linear Conditional Random Field to integrate these feature and make further inference.

• Swimming: Bächlin et al. [9] present a swimming monitoring system called SwimMaster that monitors swimming performance and technique with acceleration sensors at the wrist and at the back.

• Activities in hospital: Bardram et al. [12] propose to detect the progress of the work inside an operating room with embedded sensors and body-worn sensors. In [56],

Inoue et al. recognize nursing activities such as blood pressure measurement with mobile devices recording the acceleration data. By considering the prior probability of the activities happening in a specific time slot, they are able to outperform conventional classifiers with large margin.

• Activity recognition meets social network: While previous works recognize human activ-ities with sensor data, there are some approaches trying to recognise activactiv-ities with social network data. Zhu et al. [170] recognise activities based on the tweets from so-cial media. They crowd-label the tweets and process them into a feature vector with natural language processing methods, so that they can be fed into conventional clas-sifiers for training and testing. In [29], Du et al. propose to predict the attendance of social activity published on the website. They consider multiple features such as content, spatial and temporal context of the social event as features and use matrix factorisation for attendance prediction. Zheng et al. [167] mine location features and activity-activity correlations from the web and perform matrix factorisation for activity and location recommendation.

2.3.6 Discussion

Most of the aforementioned activity recognition techniques are inapplicable in our scenario, since most of them use static models with predefined data sources, and are not able to adapt with dynamically available sensors that may be potentially beneficial to the recognition ac-curacy. Even though the knowledge-based method can be used to deal with the unseen contexts provided by the sensors, the parameters specified by the method is general knowl-edge and cannot achieve satisfactory performance due to the fact that they usually sacrifice the recognition performance for the alleviation of labelling efforts. On the other hand, some researchers personalise the activity model to a specific user for achieving high accuracy.

However, they do not consider the dynamically available sensors that are common in real-istic scenarios. The lack of related work and the importance of addressing sensor dynamics have motivated us to propose methods for incorporating the sensors dynamically for activ-ity recognition.