3.1.1
Unobtrusive human activity classification through device-
free sensing
The focus of research in wireless networks has shifted to device-free sensing: de- tecting wireless disturbances in an environment to classify and recognize human activities. This method is known as ”device-free sensing”. The first examples from this come from the early 2010s, like research conducted by Fang et al. [33] and Gu et al. [8].
Fang et al. used the data available from a smart home (such as temperature, motion and doors opening/closing) to classify what a person was doing (bed, toilet, breakfast, computer, dinner, laundry, outdoor, lunch, taking medicine) by using the available values. These values were fed to an RBM to classify, and later compared to more classical machine learning approaches like HMM and Naive Bayes. On the other hand, Gu et al. used an access point (transmitter) and laptop (receiver) to analyse the received signal strength (RSS) of the WiFi (2.4 GHz, IEEE 802.11b) signal and other ambient WiFi signals. The focus was on the basic activities, such as standing, sitting and walking. The algorithm used to classify the data was a k-NN algorithm.
Then, in 2016, Li et al. [6] looked to modify an access point to use the channel state information (CSI) of the wireless local area networks (WLANs) to classify
DEVICE-FREE SENSING & DEEP LEARNING 3.1. STATE-OF-THE-ART the activities walking, sitting, lying, standing, squatting, falling and crawling in order to support the elderly. It also used the 2.4 GHz frequency, but instead used a Random Forest classifier. The access point was modified in such a way that it could gather the CSI from nodes (other devices) in the room and based on the changing state, the features would change. Thanks to this, the system was robust in both line-of-sight as well as non-line-of-sight situations.
Later, in 2017, Huang et al. [5] used four corner nodes and an access point to track humans in a room based on their height: the access points would transmit data to the access point over different frequencies (2.41 to 2.49 GHz) and the packet receive rates (PRR) was the feature extracted to label the data on and an SMO and k-NN were trained and compared. Also in 2017, Wang et al. [20] used eight nodes and a laptop to localize people, as well as classifying four activities and four gestures. The system would place eight nodes around a room and connect these nodes with one-another, then the LOS properties, as well the RSSI was used to train a deep learning model and classify the activities. Another interesting development in 2017, was the use of a radio waves as a radar, proposed by Haideret al. [4]. A transceiver was placed in a room and would transmit an electromagnetic wave in the 3.3 to 10.3 GHz range. The signal would interact with the environment (and thus be reflected or broken) and the transceiver would pick up the signal again and based on this learn. However, as this was a proposed system, no actual learning techniques were discussed. Lastly in 2017, Murad et al. [2] deployed a recurrent neural network (RNN) with LSTM and tested it on benchmarked data (much like Zheng et al. ). The evaluated activities included opening doors, opening fridge, opening dishwasher, opening drawers, cleaning table and toggling a switch.
However, the current state of the art shows how much research is going into the field of device-free sensing when looking at what has been concluded in the first few months of 2018 [44][45][14][15].
Booranawong et al. [44] proposed an algorithm to track and detect human movements. This proposed idea would focus on walking and moving in general and required a single base node, one receiving and three transmitting nodes that would use the 2.4 GHz frequency. All nodes were placed on the same altitude and the base station would look at the RSSI of each transmitting node. However, there was no machine learning involved, as the proposed algorithm used a threshold to detect movement.
Guoet al. [45] developed a hybrid system based on the 2.4 GHz (IEEE 802.11n) frequency to analyse the RSSI and CSI (and compare the two), as well as adding skeleton data by using a Kinect (image analysis). A total of sixteen activities were classified using k-NN, random forest and decision tree techniques. This showed that combining both methods greatly improved the accuracy of the system.
Han et al. [14] looked at a passive way to detect humans: CSI fingerprints in a room. This method used a receiving node (laptop, as the receiver) and looked at the CSI and RSSI, and was made robust in a NLOS scenario by using a sub- carrier matrix (of 30 subcarriers). The use of multiple antennas was also tested and increased the performance. The algorithm used was a self-developed voting algorithm to vote on which fingerprint it was.
Lastly, Shahet al. [15] looked at the medical world and focused on narcoplectic patients, detecting sleep attacks and sleepiness. The proposed method consisted of a receiver and transmitter that used the 2.4 GHz channel state information (CSI) to calculate the phase and amplitude, as well as using the S-Band Sensing technique.
With deep learning, Liu et al. [46] proposed a deep believe learning (DBL) based algorithm to successfully identify critical and weak links, which could in turn result in an optimization of networks. This was done by evaluating the link states and learning from this.
Earlier this year as well, Dang et al. [10] proposed a Kalman filtered CSI and PCA (principal component analysis) solution in LOS, NLOS and wall environment experiments and are compared. They used the classical machine learning technique SVM and the gathered data is matched against the data in the fingerprint database and they show an accuracy of 95%, while claiming to have the best algorithm in terms of average error and indoor activity recognition accuracy.
[11]et al. proposed an activity recognition algorithm that can be ran on com- modity Wi-Fi enabled IoT devices, thus enables the cost for dedicated devices. Their designed a novel OpenWrt-based IoT platform to collect their CSI and used an RNN to recognize and classify the data. Their results show a 97.6% accuracy for 10 volunteers. The devices used are two routers, which are still quite large compared to the potential devices. Also, a convolutional network can potentially achieve higher accuracy.
DEVICE-FREE SENSING & DEEP LEARNING 3.1. STATE-OF-THE-ART Linux CSI tool, and first use feature extraction and PCA to clean the data. They used the k=NN classifier to classify the activities in a LOS environment. There were three volunteers to gather and test data and to test against. While they have shown that the Linux CSI tool can correctly be used to gather the data, their system still needs manual feature extraction and only has three volunteers.
[?] et al. proposed a new system based on HMM. Their research focused on how to estimate the signal propagation in an adaptive complex environment. They proposed a new ambience identification method, called Ambience Sensor (Asor), in order to improve the performance of the applications on top. Furthermore, they integrated Asor into a localization method called Aloc) and claim their method is superior when it comes to the median detection rate of propagation ambience.
[18]et al. developed a device-free people counting system, based on CSI. The CSI makes sure the system is a non-image based counting system, thus meaning that no actual pictures of humans are being used, but rather the CSI data is analysed through a DNN. Their testbed showed an accuracy of 88% on average when it came to estimating a crowd size up to nine people.
Lastly, [19] Wang et al. proposed a system based on 5 GHz CSI data to create an indoor localization system. The images used were that of the angle of arrival (AOA), which were gathered in the online phase. During the offline phase, a DCNN was trained to localize the humans in two representative environments. The devices used are an access point and a laptop, both equipped with the NIC 5300, thus implying the Linux CSI Tool was used.
3.1.2
Analysing networks and their disturbances
To start with analysing networks using deep learning, the importance to this should be clear: as an attempt is made to classify human activity through measuring these networks, it is important to know if deep learning can be used for this. Luckily, Kulin et al. [35] has proven that deep learning (and specifically CNNs) can be used to detect interferences in wireless networks and can be improved using these. Secondly, Wang et al. [47] (2018) has shown that more information can be gathered from signals (and increasing the accuracy of NLOS systems) by looking at the spatial information of the environment. This was done by providing structure blocks and a coherent histogram.
Also, Avci et al. [48] has shown that a CNN can be used to analyse structural health monitoring (SHM), and especially in the damage detection (SDD).
Choi et al. [13] proposed an RNN with LSTM block to detect NLOS and LOS based on the channel state information (CSI) of links, as well as the received signal strength indication (RSSI) in a cross-layer manner to accurately figure out whether a wireless device was in direct line-of-sight or not.