Work
This chapter has presented a detailed overview of the collaborative filtering datasets under study in this work, with the aim of contrasting and comparing the basic characteristics of these datasets. The comparisons are done with the complete datasets as well as with defined portions of the datasets, called views. Standard collaborative filtering approaches, methodology and metrics are used for the comparisons in order to ensure that any differences noted in the results are due to the datasets and not to the approach used. The datasets used are freely available and have been employed in previous studies; these studies are detailed in addition to studies relating to dataset views.
From the experiments in this chapter it has been found that:
• There are different rating distributions across the four datasets; this gives an early indication that performance across the four datasets may differ. • There are variations among the “baseline” MAEs, found with a 10/90 split
of test and training data and using a standard collaborative filtering ap- proach. In particular, the MovieLens dataset has the best performance, followed by the last.fm dataset, followed by the Epinions dataset. The bookcrossing dataset has the poorest performance in this experiment. For each of the datasets, this gives a “baseline” MAE value with which to com- pare future MAE results. It leads us to expect to see a similar ordering of performance for the four datasets in future experiments.
• When changes are made to the percentage split of training and test data, it can be seen that, in all but the Epinions dataset, as the training data decreases, the MAE increases. The largest increase across different splits is seen for the last.fm dataset. The results for the Epinions dataset show no clear pattern, which gives us an early indication that results for this dataset across future experiments may not follow an expected pattern.
• When different collaborative filtering techniques are explored in addition to baseline techniques, there was no one technique which performed best across all four datasets. The technique used in this work, user-based Pearson cor- relation, performed well, and quite close to the best-performing techniques in most cases. For this reason the technique was seen as a viable approach for further experiments.
• When considering the user rating views for three of the four datasets (all but the bookcrossing dataset), the performance shown was as expected given the definition of the views, i.e., high user rating views gave the best performance and low user rating views gave the worst performance, which was worse than the baseline performance. These results are useful to check the validity of the views created and as a baseline comparison for future experiments.
• When considering three popular item views for each of the four datasets, the intuition is not as clear with respect to how performance is affected by the existence or otherwise of popular items. It was shown that, in all four datasets, it was the medium popular item views that gave the best performance. This indicates that views containing mostly unpopular or
mostly popular items do not perform as well. These results are useful in helping us understand the popular item views. In addition, the results are useful as baseline comparisons for future experiments.
The contributions of the work outlined in this chapter relate to providing an overview and analysis of the characteristics of the datasets and views used in the work and in providing baseline performance results. This contribution motivates the need to analyse the characteristics of datasets further. Future work could consider additional views and additional datasets.
The datasets and views outlined here will be considered in the subsequent chap- ters. The next two chapters will consider parameter variations in the collaborative filtering approach used and the chapters following that will consider features of the datasets and views and how these may be used for performance prediction.
Learning Neighbourhood-based
Collaborative Filtering
Parameters
4.1
Introduction
The work outlined in this chapter uses a genetic algorithm approach to learn the optimal set of parameters for a neighbourhood-based collaborative filtering approach. The motivation is to assess whether different datasets require different parameter settings. Parameters are evolved for the entire four datasets which were outlined in the previous chapter: MovieLens, bookcrossing, last.fm and Epinions.
The chapter outline is as follows: firstly, in Section 4.2 the motivations of learning neighbourhood-based collaborative filtering parameters are briefly outlined, in addition to a brief overview of genetic algorithms. Some previous work using a genetic algorithm approach within the area of collaborative filtering is also outlined. The methodology used to learn the parameter values is then presented in Section 4.3. The genetic algorithm approach, associated genetic algorithm parameters and the collaborative filtering parameters which are to be learned are outlined in this section. Results are then presented in Section 4.4 for all four datasets using the parameters outlined. The parameter values obtained by the genetic algorithm approach are evaluated using a standard training and test set and the standard memory-based collaborative filtering approach. In Section 4.4.3 the suitability of the problem to a genetic algorithm approach is considered based
on the average and best fitness found at each generation for each dataset. Finally a discussion of the results and conclusions are presented in Sections 4.5 and 4.6 respectively.