4.4 Results
4.5.2 Popularity-Focused Rating-Based Collaborative Filtering Method (PFRBCF)
we propose a way to address this problem by adding IR characteristics into the Weighted Slope- One algorithm to make it more suitable for the integrated model. As described in experiments conducted in Section 4.3, we cluster users into different topic categories based on their different
135
topic interests. Each topic category contains search behaviour information for similar taste users. For each query, we know corresponding topic category and relevance assessment. Based on the data we collected, which includes each query, its corresponding topic category, the user behaviour data in the topic category and the relevance assessment for this query, we can observe that in the corresponding topic category for each query, if any document in this topic category has been rated by most users, and also its obtained ratings are relative higher, it is more relevant to this query. From this observation, we can assume that document frequency in one topic category (how many time this document occurs in this topic category) indicates how many users in this topic category have rated this document, and that average document ratings are two factors affecting the document relevance. We propose a modified method by adding these two factors into the existing Weighted Slope-One algorithm to improve its suitability for the integrated model. This relationship between these two factors and relevance is shown in Table 4.8 in descending order, πΉ! is the document frequency and π ! is the average rating that item π
received.
ππ Β πΉ! Indicator of
Relevance
High High High Relevance High Low Less Relevance Low High Less Relevance
Low Low Non Relevance
Table 4.8 The Relationship between ππ, πΉ! and relevance
Based on this assumption described above, we propose a novel Popularity Focused Weighted Slope-One (PWS1) algorithm which uses this relationship to extend the existing rating-based WS1 algorithm. This new algorithm is expected to be more effective for our integrated model because it includes two IR system factors. Our new prediction algorithm is shown in Equation
136
(4-14). This adjusts the recommender score that user π’ will give to item π by adding a parameter
!"# !!!!
!!!!!!!
!
to the standard Weighted Slope-One.
π!"#!(π’) != π!"! π’ !β log πΉ!+ π 1 β π ! 1 + π ! (4-14)
where πΉ! is the document frequency for document π, and π ! is its average ratings, k is a constant value which is used for the condition that when this predicted documentβs frequency is only 1 in a topic category there will be a zero value for the final prediction for this document. We utilize 1 β (π ! 1 β π !) and log πΉ!+ π to control the contribution of document frequency and average rating of this document for the final prediction score.
Figure 4.4 The trend of how document frequency and document average rate affect document relevance.
Figure 4.4. shows the relationship between document frequency and document average rating and the document relevance. These two factors work in combination with the original Weighted Slope-One algorithm when predicting the potential relevance for each document to smooth the effects of individual high ratings.
137
In the next section, the effectiveness of this popularity-focused rating-based CF algorithm is examined and compared to the other existing recommender algorithms in our integrated IR model.
4.5.3 Experimental Investigation
The results presented in Tables 4.6 and 4.7 reveal that, the hybrid recommender approach which incorporates CBF results with existing CF algorithms results outperforms the method which only combines the IR output with existing recommender algorithms results. Table 4.9 shows that MAP and precision at a cut-off position for the methods used in Table 4.7, and also compares these methods with our novel one which combines the IR output with CBF and the proposed PFRBCF by CombANZ fusion operator.
MAP P@5 P@10 P@20 BL 0.1275 0.1173 0.0967 0.0680 BL+QE 0.1498 (+22.3%) 0.1360 (+15.9%) 0.0890 (-7.96%) 0.0710 (+4.41%) BL+CBCF+CBF 0.2017 (+58.2%) 0.1842 (+57.0%) 0.1400 (+44.8%) 0.0987 (+45.1%) BL+IBCF+CBF (+56.1%) 0.1990 (+38.7%) 0.1627 (+38.6%) 0.1340 (+30.9%) 0.0890 BL+TBCF+CBF 0.2057 (+61.3%) 0.1713 (+46.0%) 0.1373 (+41.9%) 0.0933 (+37.2%) BL+RBCF+CBF 0.2150 (+68.6%) 0.1687 (+43.8%) 0.1343 (+38.9%) 0.0917 (+34.9%) BL+PFRBCF+CBF (+75.0%) 0.2231 (+58.9%) 0.1865 (+57.2%) 0.1520 (+58.8%) 0.1080
Table 4.9 MAP, P@5, P@10 and P@20 value for 7 runs on hybrid approach, merged by CombANZ fusion method.
From Table 4.9, it can be seen that our proposed PFRBCF algorithm obtains the best results, increasing MAP by 75% compared to the BL run, and by 50.74% compared to BL+QE when using the CombANZ method. MAP and precision at different cut off ranks for the hybrid approach show that our method achieves the greatest improvement for P@5, P@10 and P@20
138
compared to exploiting the other four existing recommender algorithms, CBCF IBCF TBCF and RBCF, to output prediction results to be combined with the IR output in the integrated model. These results provide answers to our research question:
β’ Do existing recommender algorithms provide suitable solutions in this application?
Existing recommender algorithms can be used to benefit the integrated IR model, however, since the RSs and IR system have different goals, these existing algorithms are not ideally suited solutions in this application. According to the algorithm we proposed and the experimental results we obtained for this alogrithm, we can conclude that adding features suitable for an IR application, such as the document frequency and the itemβs mean rating (which implicitly indicates the relevance of the item) into recommender algorithms, can make them more suitable for use in the integrated model and to achieve better results compared to the use of existing recommender algorithms.
4.6 Summary
In this chapter, we reviewed a range of existing recommender methods, including content-based filtering and collaborative filtering algorithms. We investigated their effectiveness in the integrated IR model. The results obtained demonstrate that they all can function as a recommender component to successfully improve IR search results for a query by utilizing the search history of previous users. However, IR systems and RSs have different objectives, where an IR system aims to find the most relevant items for a user, whereas RSs attempt to find items of interest to a userβs tastes or interests. Based on this observation, we propose a simple modified popularity-focused rating-based recommender algorithm by incorporating the IR factors into an existing rating-based algorithm by incorporating relationship between the document frequency in
139
each topic category, document mean ratings and the document relevance grade. Results of the experimental evaluation of this new method in the integrated IR model demonstrate its effectiveness, which outperforms existing algorithms.
Besides investigating several recommender algorithms, in this chapter, we also reviewed six data fusion methods. They were examined in experiments combining the outputs of IR results and output of CF algorithm in the integrated model. The CombANZ fusion operator was found to achieve the best results. These fusion methods were also investigated in an experiment combining the outputs of IR, CBF algorithms and one of the CF algorithms. The CombANZ method again obtains the best results for these experiments.
Five existing recommender algorithms were examined for use directly to generate recommendation rank lists and in combination with the output of an IR component in the integrated model. The shortcomings of these methods were examined and a modified recommender algorithm introduced which was shown to improve the effectiveness of the integrated model. In next chapter, we aim to further examine if using recommender algorithms directly is the ideal method for the integrated model. To investigate this question, we attempt to exploit recommender techniques indirectly by using them in a cluster-based PageRank method.
140
Chapter 5
5
Exploiting Recommender Techniques
in Cluster-Based Link Analysis
The basic idea of all experiments conducted in previous chapters is to employ recommender algorithms (RAs) to generate prediction results of items of potential interest to a user and to combine these results with the output of a standard IR system. However, as we have observed in the previous chapter, recommder systems (RSs) have fundamentally different goals to IR systems, and employing existing RAs may not be the best approach in the integrated model. Based on this observation, we make a hypothesis that instead of utilizing RAs in a straightforward way, we might take advantage of RAs by using them in an indirect method to aid the IR component in the integrated model. Since we already know that the PageRank algorithm has been widely proved to be an effective method in the IR system, in this chapter, we investigate the utilization of RAs in a cluster-based PageRank model to generate recommendations for the current query, and combine these recommendations with standard retrieval results in an alternative integrated model. We also propose a novel multiple cluster model by incorporating the current query into a model which looks into both the correlation between documents and the correlation between the cluster and documents. The chapter begins by considering how to employ this method in the integrated
141
model. This is followed by a description of the details of the methods to be used in this investigation.