Home Automation with Machine Learning

Home Automation with Machine Learning for Activity Recognition Page 1

Home Automation with Machine Learning

for Activity Recognition

Mr. Shubham Kulkarni

Dept of Computer Engineering, KJEI’S Trinity Academy of Engineering, Pune Email: [email protected]

Ms. Gauri Kumbharkar

Dept of Computer Engineering, KJEI’S Trinity Academy Of Engineering, Pune Email: [email protected]

Ms. Shubhahasta Shinde

Dept of Computer Engineering, KJEI’S Trinity Academy Of Engineering, Pune Email : [email protected]

Prof. Chanchal Kedia

Dept of Computer Engineering, KJEI’S Trinity Academy Of Engineering, Pune email : [email protected]

Home Automation with Machine Learning for Activity Recognition Page 2

Abstract

- Activity recognition plays an important role in providing assistance and care for users in smart homes.

Multi-sensors can provide assistance to human by collecting the data of human activities. Using multi-sensor data in the research of pattern recognition is an area where rapid technological development is needed. Using deep learning with sensor data help us to analyze the sequence of activities recorded by a specific resident.

Here, we are using various deep neural networks (DNN) i.e. Recurrent Neural Network (RNN) and algorithms used for machine learning. Advancement and development of deep learning makes it possible to perform automatic high- level feature extraction and thus, it achieves more promising performance in many areas. Since then, deep learning based methods have been widely adopted for the multi-sensor-based human activity recognition tasks.

Keywords

–Human activity recognition, Deep learning, machine learning,Smart home, Activity of daily life (ADL).

I.

I

NTRODUCTION

In recent years, activity recognition in Smart Homes with the use of simple and ubiquitous sensors, has gained a lot of attention in the field of ambient intelligence. It also assisted living technologies for enhancing the quality of life of the residents within the home environment.

Multi-sensors can be used to provide assistance to human by collecting the data set of human activities. In this paper, we are using the concept of deep learning and various machine learning concepts. Deep learning is a general term for neural network methods which are based on learning representations from raw data and contain more than one hidden layer.

Applying machine learning (Deep Neural Network) technique to traditional home automation system. Deep learning tends to overcome those limitations. Deep learning works for human activity recognition(HAR) with different types of networks. The feature extraction and model building procedures are often performed simultaneously in the deep learning models. The features can be learned automatically through the network instead of being manually designed by using deep neural network. Besides, the deep neural network can also extract high-level representation in deep layer, which makes it more suitable for complex activity recognition tasks that are difficult to recognize. When faced with a large amount of unlabeled data, deep generative models (Hinton et al., 2006) are able to exploit the unlabeled data for model. Our goal is to make more efficient home automation system for human activity recognition by applying previous techniques and present potential future research directions.

In section 2, project pre requirements are to be given like history and existing methods. In Section 3, we discuss the pattern analysis of multiple sensor. Finally, this paper is concluded in Section 4. References are included in section 5.

II. RELATED WORK

Many researchers have made a lot of attempts on building an accurate and robust acceleration-based human activity recognition(HAR) system.

There are several neural networks used for the research.

Firstly, Artificial Neural Networks (ANN) based activity recognition in smart homes was proposed in recent years.

In this research, activity recognition models have been classified into data-driven and knowledge-driven approaches.

The data-driven approaches are capable of handling uncertainties and temporal information [5] but require large datasets for training and learning.

Home Automation with Machine Learning for Activity Recognition Page 3

Secondly used network was RNN i.e. Recurrent Neural Networks for the same research. LSTM(long-short term memory) is a recurrent neural network architecture that is designed to model temporal sequences and learn long-term dependency problems. This network is well suited for language modeling tasks; it has been shown that the network in combination with clustering techniques increases the training and testing time of the model as mentioned [3] and outperforms the large scale acoustic model in speech recognition systems [4].

Recent years have witnessed the fast development and advancement of deep learning. Deep learning is the subpart of machine learning. Machine learning is a branch of "artificial intelligence". In the field, aiming to automatically analyzes the law from the data, and use the algorithm to predict the unknown information.

Deep Neural Networks (DNN) is developed from artificial neural network (ANN). Traditional ANN often contains very few hidden layers while DNN contains more hidden layers.

With more layers, DNN is more capable of learning from large data. DNN usually serves as the dense layer of other deep models. For example, in a convolution neural network(CNN),several dense layers are often added after the convolution layers.Here we mainly focus on deep neural networks (DNN).

Fig.1. Previous System III. OVERVIEW OF ARCHITECTURE

Fig.2. System Architecture

Android smartphone is used as a user control device;

end users use this to control various devices. The Raspberry Pi is a credit card sized single-board computer with an open-source platform that has a thriving community of its own, similar to that of the Arduino. It can be used in various types of projects from beginners learning how to code to hobbyists designing home automation systems.

In this project, Raspberry Pi is used as the master device.

IV. PATTERN ANALYSIS OF MULTI-SENSOR To validate our proposed idea, we are collecting the data from various sensors that are used in smart homes.

1. Data collection using multiple sensor

A combination of sensors consists of pressure sensors, light sensors, temperature sensors, sensing sensors and PIR sensor. Once these sensors are used, the main control server will be countered over time. If events are measured in time by recording activation, it could be marked as 1, otherwise 0.

2. Relationship between sensors and activities

Home Automation with Machine Learning for Activity Recognition Page 4

This paper aims to conserve energy by predicting the usages of the sensor depending on the relation of sensor and time duration.

Consider the case, that the subject is watching TV in the living room and the sensor on the air conditioner is repeatedly used. In this case in terms of energy, it is not good to turn ON and OFF device frequently. Hence, if the control system understands this patterns , it can save energy by not turning OFF the device if it is to be used again. ON and OFF of the switches can be controlled by smart devices which has bluetooth, wifi ,etc.

In other case, if there are some repeated activities or behaviours that uses repetitive sensors, the system can provide the service in advance. For example, when the subject or user enters the room automatically the fans are switched ON or the lights turn ON.

Fig.3. Sequence Diagram V. ACTIVITY RECOGNITION

In this section we provide the details of the proposed model. Activity recognition using deep neural networks referring various models used in this system. First model used is LSTM model. The LSTM model used for activity recognition describes the usage of multiple sensors for input parameters and analyzed the relationship between the sensors.

Considering this, we also used Residual RNN structure. It is known as highly efficient analysis of sequential data with minimal network layer, because residual vectors can reduce computation. We adapt the RNN structure because the input 46parameter of multiple sensors is in time series. Secondly, we applied the GRU model.

A] APPLYING FOR LSTM MODEL

Our proposed model consist of Residual-LSTM and GRU which are widely used in RNN.The LSTM cell has essentially three or four gate structures, and they assist to control the flow of information into or out of their memory. In our proposed model, we utilized the four gates, which are the input gate, output gate, forget gate, and input modulation gate.

Fig.4. LSTM Cell Structure

B] ANALYSIS OF DATA USING RESIDUAL-RNN The Residual-RNN has better performance in analyzing the input sensor data than RNN models. For example, a general LSTM structure can keep a temporal gradient flow without attenuation by maintaining forget gate. Although this gradient flow is not related to the relevant input data, it leaks into next layer without any conditions.

Home Automation with Machine Learning for Activity Recognition Page 5

Fig.5. Residual-LSTM

V. IMPLEMENTATION DETAILS

We implemented our models using Python and Tensorflow library. Our model consist of five modules they are – 1] basic home automation, 2] LDR module, 3] PIR sensor, 4]

Raspberry Pi, 5] WSN.

Softwares used in our system are python , Arduino IDE, Tensorflow, software tools, etc.

VI.RESULTS

START SWITCHING TO MANUAL MODE

Home Automation with Machine Learning for Activity Recognition Page 6

LIGHTS ON LIGHTS OFF

A

UTOMATIC

M

ODE

V.CONCLUSION

In this paper, we concluded with the implentation and results of our system. Compared to traditional pattern recognition methods, deep learning reduces the dependency on human- crafted feature extraction and achieves betterperformance automatically learning high-level representations of the sensor data.

V.FUTURE SCOPE

Furthermore,we are trying to apply attention module that eliminates meaningless data and produces better performance.

Extensive results demonstrate that Residual-RNN significantly outperforms the state-of-the-art competitors. Especially, our models well deals with a high accuracy rate than others.

However, it tended to take a little more time compared to others. Also, if the amount of datasets was a bit more, we can find better results. Design issues can be solved by learning new concepts related this topic.

VI.REFERENCES

1. Inoue, M., Inoue, S., Nishida, T., 2016. Deep recurrent neural network for mobile human activity recognition with high throughput. arXiv pre

printarXiv:1611.03607 .Jiang, W., Yin, Z., 2015.

Human activity recognition using wearable sensors by deep convolutional neural networks, in: MM, ACM. pp. 1307–1310.

2. Sundermeyer, M., Schlüter, R., Ney, H.: Lstm neural networks for language modeling.In: Interspeech, pp.

194–197 (2012)

3. S. Chen, T. Liu , F. Gao, J. Ji, Z. Xu, B. Qian, H. Wu and X.Guan, "Butler, not servant: a human-centric smart home energy management system," IEEE Communications Magazine, vol. 55,no. 2, pp. 27-33, 2017.

4. 19. Sak, H., Senior, A., Beaufays, F.: Long short- term memory recurrent neural network architectures for large scale acoustic modeling. In: Fifteenth Annual Conference of the International Speech Communication Association (2014)

5. Khan, A., Mellor, S., Berlin, E., Thompson, R., McNaney, R., Olivier, P., Plötz,T., 2015. Beyond activity recognition: skill assessment from accelerometer data, in: UbiComp, ACM. pp. 1155–

1166.

6. Yuen, J., Torralba, A.: A data-driven approach for event prediction. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6312, pp. 707–720. Springer, Heidelberg (2010).

doi:10.1007/978-3-642-15552-9 51

7. Kasteren, T.L., Englebienne, G., Kröse, B.J.: Human activity recognition from wireless sensor network data: Benchmark and software. In: Chen, L. (ed.) Activity Recognition in Pervasive Intelligent Environments, pp. 165–186. Atlantis Press, Amsterdam (2011)

8. Ronao, C.A., Cho, S.B., 2016. Human activity recognition with smartphone sensors using deep learning neural networks. Expert Systems with Applications 59, 235–244.

9. Almaslukh, B., AlMuhtadi, J., Artoli, A., 2017. An effective deep autoencoder approach for online smartphone-based human activity recognition.

International Journal of Computer Science and Network Security 17, 160.