Machine learning techniques for load disaggregation

The last stage in a NIALM method is the implementation of a disaggregation algorithm to separate individual appliance loads from the overall signal. There are many types of disaggregation algorithm that can be clustered under the umbrella of machine learning methods. Machine learning, in the context of data mining, explores the study and construction of algorithms which

Electrical Signature Steady State Transient State Time Domain [111, 119] Frequency Domain [113, 114] Time Domain [115, 120] Frequency Domain [117, 118]

Figure 2.1: Proposed electrical signature classification

can learn from and make predictions on data. According to the literature [84, 99, 121–123], machine learning methods can be divided into two main groups, supervised and unsupervised learning techniques.

Supervised machine learning techniques require labeled data for training the classifier so that it can recognise the appliances from the aggregated data. The extracted features are matched with a database of load signatures already available in order to identify an event associated with an operation of an appliance. The goal of supervised machine learning techniques is to approximate the mapping function, Y = f(X), between input variables, (x), and an output variable, (Y), so that a new input data, (x’), can be predicted from the output variable, (Y’). This prediction can be based on optimization methods which minimise the error function3by systematically choosing input values or by pattern recognition methods which recognise the patterns and regularities in the input data [84].

Supervised learning problems can be further grouped into regression problems where the output variable is numerical or quantitative, such as the size or the temperature value and classification problems, and where the output variable is categorical, such as “yes” and “not”, or “green” and “blue”.

Unsupervised machine learning techniques, on the other hand, do not require any previous training, so the need for data training can be eliminated. Unlike most of the supervised load disaggregation approaches, the unsupervised methods are non-event-based. The goal for unsu- pervised learning techniques is to model the underlying structure or distribution function, f(X), of the input data, but in this case without knowing the corresponding output variable, (Y). Unsupervised learning problems can be further grouped into clustering problems where the goal is to discover the inherent groupings in the data, such as grouping customers by purchasing behaviour and associated problems, where the goal of the rule learning is to discover rules that describe large portions of the data, such as people who buy X also tend to buy Y.

In [44],an unsupervised approach to determine the number of appliances in the household is proposed. The method creates clusters of the steady-state power consumption of the appliance changes and then employs a matching algorithm to reconstruct the original power signals using them. The method was successful when disaggregating large appliances, but presented difficulties with the small ones.

In [124] the temporal ordering implicit in on/off events of devices to uncover motifs (episodes) corresponding to the operation of individual devices are extracted and then subjected to a sequence of constraint checks to ensure that the resulting episodes are interpretable. The preliminary results of the study showed the capabilities of the model in distinguishing devices with multiple power levels.

categories, the different types of measurements, sampling frequencies, and feature selections. Most machine learning approaches, nevertheless, are based on supervised techniques and few unsupervised techniques are reported for NIALM tasks. In [13] supervised and unsupervised NILM techniques are compared, results show the former to be more accurate with errors around 2-5%, while the latter are less accurate (5-15% errors). The use of unsupervised methods can, therefore, be explored for residential environments where there are usually only tens of different loads with predictable signatures, but these methods do not seem appropriate for commercial buildings.

2.3.4.1 Algorithms for supervised machine learning

Algorithms for supervised machine learning can be divided into parametric and non-parametric algorithms, according to the degree of assumptions taken in the learning process [125]. In the former, assumptions about the learning process are made and these can greatly simplify the learning process, but they can also limit what can be learned. Non-parametric machine learning algorithms, on the other hand, do not make strong assumptions as they are free to learn any functional form from the training data. Non-parametric methods are often more flexible and achieve better accuracy, but they require much more data and training time. Examples of non-parametric algorithms include Decision Trees, Naive Bayes and K-Nearest Neighbors. The Decision Trees algorithm performs classification in two initial phases and is evaluated in a third one. The first phase is the tree building, or growth phase, in which the tree is built by recursively splitting the data into two or more branches. The value of splitting points depends upon how well separated, or “pure”, the differences are between appliance signatures [121]. In the second phase, or tree pruning phase, the algorithm keeps growing by splitting nodes as long as the new splits increase the branches “purity”. The process makes use of the training

data set for optimisation, by eliminating any leaf that increases the error rate4[126]. Finally, in the performance evaluation phase, once the tree is fully grown and then pruned, the decision tree model can be used to predict the class value for new patterns. In the evaluation stage, the prediction accuracy of the decision tree classifier is evaluated using the training data set. The 10-fold cross-validation and the leave-one-out cross-validation are standard validation methods [127]. Decision tree algorithms are fast at learning and making predictions and can achieve high levels of accuracy for a broad range of problems without the requirement for any special data pre-processing, thus providing high transparency within the classification process. In [128] a building energy demand predictive model, based on the decision tree method and which is able to classify and predict categorical variables, is developed. The advantage of the model over other widely used modelling techniques lies in its ability to generate accurate predic- tions with interpretable flowchart-like tree structures that enable users to quickly extract useful information. The method has been applied to estimate residential building energy performance indices by modelling building energy use intensity levels. The results demonstrate that the use of the decision tree method can classify building energy demand loads accurately, at 93% for training data and 92% for test data.

Naive Bayes are statistical learning algorithms for predictive modelling comprised of two types of probabilities that can be calculated directly from the training data [129], the probability of each class and the conditional probability for each class given each input value.

In [130], a naive Bayes classifier was used to detect state change and identify individual devices. The approach assumed that each device’s state was completely independent of the other devices, so that devices such as TVs and DVD players, with their highly-correlated operation, were difficult to disaggregate. Naive Bayes methods are called naive because they assume that each input

variable is independent, this assumption is often unrealistic for real data.

In a K-Nearest Neighbors model predictions are made for a new data point by searching through the entire training set for the most similar K instances (the neighbours) and summarizing the output variable for those K instances. Saitoh et al. [131] reported on the use of the K-Nearest Neighbors model for the identification of 35 appliances sampled at 4.4 kHz from which nine current-based features were extracted. For each feature the observed values, or input variables, were normalised using the average and standard deviation and a disaggregation accuracy of 80.5% was achieved. To determine the similarity between values the Euclidean distance between each input variable was used. However, in very high dimensions (a great number of input variables), results can negatively affect the performance of the algorithm and require much memory space to store all the data. To reduce this dimensional problem, only the most relevant variables can be used, however, this will affect the accuracy of the model predictions [132].