6.3 Mixture Models in Full Bayesian Framework
8.5.2 Movie Recommendation
Secondly, we demonstrate the performance of IHRM on the MovieLens data, which con- tains movie ratings from a large number of users (Sarwar et al., 2000). The task of the experiment is to evaluate the proposed inference methods. In the MovieLens data, there are two entity classes (User and Movie) and one relationship class (Like: users like movies). The User class has several attribute classes such as Age, Gender, Occupation. The Movie class has attribute classes such as Published-year, Genres and so on. The
134 CHAPTER 8. INFINITE HIDDEN RELATIONAL MODELS
Table 8.1: Performance of IHRM on MovieLens data.
CRPGS TSBGS TSBMF EA Pearson Given5 65.13 65.51 65.26 63.91 57.81 Given10 65.71 66.35 65.83 64.10 60.04 Given15 66.73 67.82 66.54 64.55 61.25 Given20 68.53 68.27 67.63 64.55 62.41 Time(s) 164993 33770 2892 - - Time(s/iter.) 109 17 19 - - #C.u 47 59 9 - - #C.m 77 44 6 - -
relationship class Like has an auxiliary attribute R with two states: R = 1 indicates that the user likes the movie and R = 0 indicates otherwise. The IHRM model for the movie recommendation system is shown as Figure 8.4. In the data set, there are totally 943 users and 1680 movies. The ratings are originally recorded on a five-point scale, ranging from 1 to 5. We transfer the ratings to be binary, yesif a rating is higher than the average rating of the user, and no otherwise. The performances of all inference methods are analyzed from 3 points: prediction accuracy, convergence time and clustering effect. To evaluate the prediction performance, we execute 4 sets of experiments with respectively 5, 10, 15 and 20 randomly selected ratings as the known ratings, and predict the remaining ratings for each test user. These experiments are referred to as given5, given10, given15 and
given20. For testing, the relationship is predicted to exist (i.e. R = 1) if the predictive probability is larger than a threshold ε= 0.5, andnonexist (i.e. R = 0) otherwise.
(a) (b) (c)
Figure 8.8: (a) The traces of the number of User clusters for the runs of two Gibbs samplers. (b) The trace of the change of variational parameter matrix ηu for the run of the mean field method. (c) The sizes of the largest User clusters of the three inference methods.
We evaluate the following four inference methods: Gibbs sampling with Chinese restaurant process (CRPGS), Gibbs sampling with truncated stick-breaking (TSBGS), and the corresponding mean field method (TSBMF) as well as the empirical approxima- tion method (EA). In TSBGS and TSBMF, the truncation parameters Ku and Km are initially set to be the number of users and the number of movies, respectively. For TS-
8.5. EXPERIMENTAL ANALYSIS 135
BMF we consider α0 ∈ {5,10,100,1000}, and obtain the best prediction when α0 = 100. For CRPGS and TSBGS α0 is set to 100. For the variational method, the change of variational parameters between two iterations is monitored to determine the convergence. For the Gibbs samplers, the convergence was analyzed using three measures: Geweke statistic on likelihood, Geweke statistic on the number of components and autocorrela- tion. Figure 8.8 shows the traces for the runs of the 3 inference methods. (a) shows the traces of the number of User clusters for the runs of the 2 Gibbs samplers. (b) shows the change of variational parameters ηu in the variational method. Table 8.1 shows that the blocked Gibbs sampler TSBGS converges approximately by a factor 5 faster than the CRPGS sampler. The mean field method TSBMF is again by a factor around 10 faster than the blocked Gibbs sampler TSBGS and thus almost two orders of magnitude faster than CRPGS. CRPGS is much slower than the blocked Gibbs sampler mainly due to the large time cost per iteration shown as Table 8.1. The reason is that CRPGS samples the hidden variables one by one, which causes two additional time costs. First, the ex- pectations of attribute parameters and relationship parameters have to be updated when sampling each user/movie assignment. Second, the posterior of hidden variables have to be computed one by one, thus we can not use fast matrix multiplication technology to accelerate the computation.
The prediction results are shown in Table 8.1. All methods under consideration achieve comparably good results. The best results are achieved by the two Gibbs sampling meth- ods. To demonstrate the performance of IHRM, we also implement a Pearson-coefficient based collaborative filtering method (Resnick, 1994). It is clear that IHRM outperforms the traditional CF method, especially when there are few known ratings for the test user. IHRM provides cluster assignments for all entities involved, in our case for the users and the movies. The columns #C.u and #C.m in Table 8.1 denote the numbers of clusters for User class and Movie class, respectively. The Gibbs samplers converge to 47-59 clusters for the users and 44-77 clusters for the movies. The mean field method has a tendency to converge to a smaller number of clusters with the same value of α0. Further analysis shows that the clustering results of the three methods are actually similar. First, the sizes of most clusters generated by the Gibbs samplers are very small, e.g. there are 72% (75.47%) user clusters with less than 5 members in CRPGS (TSBGS). Figure 8.8(c) shows the sizes of the 20 largest User clusters of the three methods. Intuitively, the Gibbs samplers tend to assign the outliers to new clusters. Second, we compute the rand index (0-1) of the clustering results of the methods, the values are 0.8071 between CRPGS and TSBMF, 0.8221 between TSBGS and TSBMF, which also demonstrate the similarity of the clustering results between Gibbs samplers and mean field method. Table 8.2 shows the movies with highest posterior probabilities in the 8 largest clusters generated by CRPGS. The values in parentheses, e.g. 161/207, means: the number (167) of coincident movies assigned to the cluster in the last 10 iterations and the average size (207) of the cluster.As we can see from the numerical values, there is quite some fluctuation in the cluster assignments. In cluster 1 most movies are very new and popular (the data set was collected from September 1997 through April 1998). Also they tend to be romance and comedy movies. Cluster 2 includes many old movies, or movies produced by the non-USA countries, or drama movies. Cluster 3 contains many comedies andcluster 4
136 CHAPTER 8. INFINITE HIDDEN RELATIONAL MODELS
Table 8.2: Clustering result of CRP-based Gibbs sampler on MovieLens data.
Cluster 1 (161/207) Cluster 2 (76/113)
My Best Friend’s Wedding (1997) G.I. Jane (1997) The Truth About Cats and Dogs (1996) Phe- nomenon (1996) Up Close and Personal (1996) Tin Cup (1996) Bed of Roses (1996) Sabrina (1995) Clueless (1995)...
Big Night (1996) Antonia’s Line (1995) Three Colors: Red (1994) Three Colors: White (1994) Cinema Paradiso(1989) Henry V (1989) Jean de Florette (1986) A Clockwork Orange (1971) Citizen Kane (1941) Mr. Smith Goes to Washington (1939)...
Cluster 3 (49/98) Cluster 4 (32/51)
Swingers (1996) Get Shorty (1995) Mighty Aphrodite (1995) Welcome to the Dollhouse (1995) Clerks (1994) Ed Wood (1994) The Hudsucker Proxy (1994) What’s Eating Gilbert Grape (1993) Groundhog Day (1993)...
Event Horizon (1997) Batman and Robin (1997) Escape from L.A. (1996) Batman Forever (1995) Batman Returns (1992) 101 Dalmatians (1996) The First Wives Club (1996) Nine Months (1995) Casper (1995)...
Cluster 5 (16/27) Cluster 6 (9/15)
Conspiracy Theory (1997) The Game (1997) Air Force One (1997) Ransom (1996) The Rock (1996) Primal Fear (1996) Crim- son Tide (1995) In the Line of Fire (1993) The Abyss (1989)...
Brave Heart (1995) Forrest Gump (1994) Fugitive (1993) Termina- tor 2: Judgment Day (1991) Indi- ana Jones and the Last Crusade (1989) Die Hard (1988) Aliens (1986) Terminator (1984) Return of the Jedi (1983)
Cluster 7 (8/13) Cluster 8 (3/6)
Shawshank Redemption
(1994) Wrong Trousers (1993) Schindler’s List (1993) Silence of the Lambs (1991) One Flew Over the Cuckoo’s Nest (1975) Godfather (1972) Rear Window (1954) Casablanca (1942)
Star Wars (1977) Star Wars: The Empire Strikes Back (1980) Raiders of the Lost Ark (1981)
8.5. EXPERIMENTAL ANALYSIS 137
Table 8.3: An example gene.
Attribute Value
Gene ID G234070 Essential Non-Essential
Class 1, ATPases 2, Motorproteins Complex Cytoskeleton
Phenotype Mating and sporulation defects
Motif PS00017
Chromosome 1
Function 1, Cell growth, cell division and DNA synthesis 2, Cellular organization
3, Cellular transport and transprotmechanisms Localization Cytoskeleton
consists of comedy and sci-fi movies. In cluster 5 all the movies are relatively new and most movies include conspiracy and government. In cluster 6 all the movies belong to the genre of action/thriller (except for Forrest Gump). Cluster 7are drama movies. The three movies in cluster 8are relatively old (from 1977 to 1981) and the main actor in the three movies is Harrison Ford. Overall we were quite surprised by the good interpretability of the clusters.