Evaluation Experimental in E-ARCS

This section will evaluate of the previous experimental studies related with adding and retrieving comments in social media.

2.4.1 How useful are Your Comments

Research into sentiment classification and opinion mining, for example [7, 184]

involves the issue of automatically allocating opinion values, for example, ‘positive’

versus ‘negative’ versus ‘neutral’ to documents or subjects using many different text orientated and linguistic qualities. Research that has been done on this topic recently, utilises SentiWordNet [62] to make classifying easier and more successful. The problem setting used in this research however is different from our work as they investigate feedback comments from the community instead of attempting to predict the feelings behind the actual comments. Much research has been done in classification that uses probabilistic and discriminative models [8] together with learning regression and ranking functions [185, 189]. The SVM Light software package [180], which is very commonly used, provides many different types of parameterizations and variations of SVM training (for example, binary classification, SVM regression and ranking and transductive SVMs, etc.).

In this particular paper, the researchers used these techniques in a new context to automatically classify comment acceptance. Kim et al [181] classify product reviews with respect to how helpful they are when involved with various textual features and Meta data. However, their best results came from using a mixture of information

29 gathered from the star ratings (for example, deviation from other ratings) supplied by the writers of the reviews; information like this cannot be obtained for all websites and more specifically, cannot be gathered for comments posted on YouTube. Weimer et al [7] use something very similar to automatically estimate the value of posts on Nabble.com (a software internet based forum). Liu et al [182] give details about a strategy for aggregating ratings for product characteristics,; making use of helpfulness classifications based on a predetermined ground truth and then measuring it against their summarisation with certain ‘editor reviews’ on these websites. The study also used community feedback to gather data and ground truths for classifying and regression [187].

Also they used tags and visual elements together with favourite assignments in Flickr to categorise photos and put them into order with regards to how attractive they are. In comparison with other work that has been done in the past, our paper is the first to utilise and assess automatic classification techniques for accepting comments in YouTube. Moreover, they were pioneers in giving a comprehensive analysis of how the comment ratings in YouTube are allocated. Both quantitative and qualitative studies were included as were dependencies on comment sentiment, the difference for ratings between categories and polarizing content.

2.4.2 News Comments

Past research has looked into the different features of the comment space dynamics.

Mishne and Glance [193] researched into weblog comments and found that they were very beneficial for improving retrieval and for discovering disagreements in blog posts.

30 Duarte et al. [62] defined blogosphere access patterns from the blog server point and discovered three sets of blogs by utilising the amount of posts there were over comments. Kaltenbrunner et al. [190] assessed how long it took the community to respond with regards to comments on Slashdot stories, and found that there were frequent temporal patterns in the way that people commented. Lee and Salamatian [196]

state that the number of comments in a discussion thread is indirectly proportional to how long it stays on the internet for. They found this out after researching into the clustering threads of two internet discussion forums and a social networking site. Schuth et al. [195] investigated into the comments on news stories on four Dutch media websites. They give details about the people who comment and come up with a technique for deciphering the discussion threads from the comments. De Choudhury et al. [8] categorise discussions on internet media with respect to how interesting they are.

The study investigates the commenting space on news articles on the internet and model the commenting behaviour for numerous sources of news. Research done in the past has found that the distribution of comments in blog posts is controlled by Zipf’s law [191, 192]. Lee and Salamatian [196] utilise a Weibull distribution for modelling the comments made in discussion threads. Kaltenbrunner et al. [62] highlight arguments for using the log-normal instead of the Zipf distribution for modelling; they utilise four different log normal to model the reaction times on Slashdot stories. Ogilvie [195]

model the distribution of comment counts in RSS feeds with the use of the negative binomial distribution. Tsagkias et al. [194] utilise a similar strategy to model comments made on news stories for prediction before it is published. Wu and Huberman [194]

31 discovered that digs can be modelled using the log-normal distribution and Szab´o and Huberman model popularity growth of online content utilising a linear model.

2.4.3 Ranking Comments on the Social Web

Various new studies have investigated barriers to the value of user supplied content, such as the value of user supplied tags [15] blog comments [16] user supplied answers on question-answer forums [17] and product reviews on the Amazon website [18] etc.

The majority of the time, these quality assessments depend on experts outside of the social internet community, for example a panel of experts can decide if a blog comment is ‘spam’ or is ‘not-spam.’

2.4.4 What Makes Conversations Interesting

Social Media Communication Analysis: Much work has been carried out into assessing discussions or comments left on blogs [62,117] and analysis has also been done on the use of this communication for estimating; how users will behave, sales and the movements of the stock market etc [114,116]. In [116] the study assessed the dynamics of communication (of conversations) in a technology blog and then used this to estimate stock market activity. However, in previous work, the relationship or effects of a particular conversation property, with regards to other features of the media object, has not been taken into consideration. In this paper researchers categorise the outcomes of conversations based on the effects of the themes and the communication characteristics of the users.