Measuring the reputation in user-generated-content
systems based on health information
Leila Weitzel
1*, José Palazzo M. de Oliveira
1and Paulo Quaresma
2 1 Universidade Federal do Rio Grande do Sul, Brasil.2 Universidade de Évora, Portugal.
lmartins@ufpa.br, palazzo@inf.ufrgs.br, pq@uevora.pt
Abstract
The Web 2.0 is the ultimate manifestation of User-Generated Content systems. Such systems contain information of different quality levels. This paper proposes how to measure reputation from social interactions existing in social networks built according to health information. We selected Twitter as a case study, since retweet function can be compared with spreading information mechanism. We provided a new methodology to rank trustworthy sources in a network based on retweet ties which can help authorities and experts about health information on the internet. Additionally, as a secondary contribution, we also perform a study about the scale free characteristics of the explored retweet network.
Keyword: Social Media, Social Network Analysis, Reputation, Information Source.
1 Introduction and Motivation
The Web 2.0 is the ultimate manifestation of user-generated content, and therefore, it is more than a set of new technologies and services. The Web 2.0 allows users to interact and collaborate with each other in a social communication environment as creators (prosumers) of user-generated content. The Web 1.0 is about connecting computers and making information available, while Web 2.0 is about connecting people and facilitating new kinds of collaboration (Kochanek, Xu, & Murphy, 2011).
The Internet can be an important source for people looking for healthcare information and the number of such internet users has often increased. Notwithstanding, how shall we know which sources are useful? How shall we be able to separate the bad sources from the good ones (Hu & Shyam Sundar, 2009; Hughes, Joshi, Lemonde, & Wareham, 2009; Stvilia, Mon, & Yi, 2009; Weaver, Thompson, Weaver, & Hopkins, 2009). Due to the global characteristics of the Internet, there is no
* Corresponding author: leila.weitzel@gmail.com, Leila Weitzel.
Volume 29, 2014, Pages 364–378
ICCS 2014. 14th International Conference on Computational Science
publisher that gives guarantee for the contents of the page; therefore, anyone can post anything on the web, regardless of his or her background or medical qualifications.
Many concerns have arisen about the quality of health information consumed online, and its detrimental effects (Gil & Artz, 2011). Although, recently many initiatives have emerged to support users that are looking for health information, e.g., many portals use ethical codes, and thus, there is the concern that such initiatives are either ineffective or even counter-productive (Bernstam, Sagaram, Walji, Johnson, & Meric-Bernstam, 2005; Eysenbach, 2002).
Health information is determinant to health-related decisions (Kumar, 2011). Patients frequently complain that information derived from traditional venues does not meet their needs. Driven by this need, many, if not the most of users, proactively seek information on their own to obtain answers instead of waiting for their next appointment with the doctor (Boberg et al., 2003). In general, the users want to know about symptoms, lifestyle factors (e.g. diet, exercise) that can affect the route of the disease, or if additional tests and treatment is necessary (Wang & Liu, 2007). Since, users act without professional guidance, they may not have sufficient knowledge and training to assess the quality of health web content (Eysenbach, 2002).
In this context, the main question here is how reliable the health online information is? Sabater and Sierra (Sabater & Sierra, 2005) pointed out that there are many significant factors that affect how users determine trust, e.g., direct experiences and witness information (also called word-of-mouth).
Therefore, the purpose of this paper is to provide an approach to assess reputation from source information in the medical domain. For this purpose, we considered reputation as a measure of quality of a supposed “reliable source” we used and adapted the available methods, by combining some metrics which are frequently used in Social Network Analysis (SNA) to evaluate reputation in other contexts, since to the best of our knowledge, it is the first research that uses SNA to evaluate user reputation in the medical domain.
The outline of this paper is as follows: In section 2, we establish the problem and we present the inherent difficulties to measure such reputation in the medical domain considering online information; Section 3 presents a brief discussion about concepts in Social Network Analysis. Section 4 we present the main features of Twitter for our study, Section 5 presents the related works. Section 6 explores the topological characteristics of social network based on retweet. Our data suggests characteristics of a scale free network. Moreover, in this same section, we present our general formulation to calculate the rank reputation of an individual in the network; Section 7 presents our main results about rank reputation of the social network based on retweet studied in our work. Section 8 provides some conclusions of our results.
2 Problem statement
Internet is frequently used as a source of information on health issues and, the most frequent reasons to visit medical websites is looking for information about symptoms, diseases, or treatment (de Boer, Versteegen, & van Wijhe, 2007). Health information available on the internet can be relatively different from each other. These vary from academic sites, peer-reviewed journals, governmental sites, health-institutions sites and individual contributions. There is also a large number of industry-related Web sites disseminating information or selling products or services in a variety of ways. When we use the Internet in order to obtain health and medical information, serious issues must be considered. Information obtained from Internet can bring improvements on health care but can also do harm if it is wrong or even misused. By looking on the bright side, health information is intended to extract discussion and communication between patient and the primary care physician (Anderson, 2004). By its very nature, i.e., the freedom for publishing, as a consequence, anyone can publish anything. Hence, there is a risk that such information, due to unawareness or bias, may be inaccurate
or ambiguous. In order to tackle this issue, several studies have proposed quality-rating instruments based on code or technical criteria to evaluate (or categorize) health web sites.
The Agency for Health Care Policy and Research (http://www.ahrq.gov) - AHCPR proposed a “Code of Ethics”. The Codes set out ethical concepts that inform the processes of self-assessment and compliance based on interpretation and specification: Credibility: It must be included the source, currency, relevance/utility, and editorial review process for the information; Content: The content must be accurate and completed, and the site must have an appropriate disclaimer that the information available may be incomplete; Disclosure: The purpose of the site must be informed to users, as well as any profiling or collection of information associated with the site; Links: It must be evaluated according to selection, architecture, content, and back linkages; Design: encompasses accessibility, logical organization (navigability), and internal search capability; Interactivity: includes feedback mechanisms and ways for the exchange of information among the users; Caveats: It must be very clear whether the function of website is to market products and services or is a content provider of primary information. The Health on the Net Foundation (HON - http://www.hon.ch) also proposed a certification based on a standard conduct code so-called Net Code of Conduct - HONcode. The code is intended to allow websites to publish more transparent information. The principles of HONcode are: Authoritative: Indication of the qualifications of the authors; Complementarity: The information should support, not replace, the doctor-patient relationship; Privacy: It must respect the privacy and confidentiality of personal data submitted to the site by the visitor; Attribution: To cite the source(s) of published information; Justifiability: The website must back up the claims corresponding to benefits and performance; Transparency: To have an accessible presentation, and email contact; Financial: It must disclosure the identifies of the funding sources; Advertising policy: It must clearly distinguish the advertising from editorial content. Consumers can access online health information directly from credible scientific and institutional sources (e.g. Medline, Healthfinder, which have HONcode) as well as unreviewed sources of unknown credibility (e.g. well-informed individuals along with quacks and charlatans), and that is the problem. It should be borne in mind that the task is a particularly complex one. The users must take into account all the standard conduct code (HONcode), in order to carry out a thorough analysis to assess trust. However, an individual might not be willing to spend the time and instead make a naïve assessment to arrive at the conclusion whether or not that the information has credibility.
It's hard for someone who is not a medical professional to make sure that a site provides reliable health information. There are many questions about reputation that users continuously have to ask: is there any evidence that the author of the Web information has some authority in the field about which she or he is providing information? What are the author’s qualifications, credentials and connections to the subject? With what organization or institution is the author associated? Are there clues that the authors are biased? Let us suppose that you read an article about Alzheimer written by a doctor on his blog. If you asking about the source of this article, you may give different answers: “a doctor,” “a blog,” “a doctor’s blog,” or just “the Internet.” Indeed, evaluating sources when doing research can be a complex task, even worse when many searching engines (such as Google, Bing and etc..) rank material according to their “idea of relevance”.
3 Social Network Analysis SNA
SNA is the mapping and measuring of relationships and flows between people, groups, organizations, computers, URLs, and other connected information/knowledge entities (Jøsang, Ismail, & Boyd, 2007; Wasserman, 1999; Watts, 2004). A social network is a social structure (a graph of a more formal point of view) between actors, mostly individuals or organizations.
The recent proliferation of web applications and mobile devices has made online Social Network - SN more accessible than ever before. People connect with each other beyond geographical and
timeline barriers, diminishing the constraints of physical boundaries in creating new ties. These ties can characterize any kind of relationship, friendship, authorship, etc. Users in network publish their profile, express their interests in any content, and moreover create links to any other users that are associated along time (Barabási, 2003). Online Social Network (OSN) provides a basis for maintaining social relationships, for finding users with similar interests, and for discovering content and knowledge (Antheunis, Valkenburg, & Peter, 2010; Shalizi & Thomas, 2011). Numerous OSN have emerged, including networks of professionals (e.g., LinkedIn), networks of friends (e.g., MySpace, Facebook, Orkut), and networks for sharing specific types of content such as short messages (e.g., Twitter), diaries and journals (e.g., LiveJournal), photos (e.g., Flickr), and videos (e.g., YouTube).
SNA focuses on understanding the nature and consequences of ties between individuals or groups, i.e. social network. SNA is widely used in social and behavioural sciences, as well as in political science, economics, organizational science, and industrial engineering (Borgatti, Mehra, Brass, & Labianca, 2009; Bruns, 2009; Crandall et al., 2010; Dana & Loewenstein, 2003; Dorogovtsev & Mendes, 2003; Easley & Kleinberg, 2010; Wasserman, 1999; Watts & Strogatz, 1998). The ties (edges) analysis is the one of several objectives of SNA, depending on the topic of interest: Psychologists and sociologists have studied friendship ties (Balkundi & Kilduff, 2005; L. C. Freeman, 1996; Granovetter, 1973; Ibarra, 1992; McPherson, Smith-Lovin, & Cook, 2001), lately other studies attempt to identify communities of users with similar interests, and within such communities they try to identify the most “influential” users. Generally speaking, the aim is to aid business and marketing process (Ahn, Han, Kwak, Moon, & Jeong, 2007; Cai, Shao, He, Yan, & Han, 2005; Cha, Haddadi, Benevenuto, & Gummadi, 2010; Cosley, Huttenlocher, Kleinberg, Lan, & Suri, 2010; Jianwei, Lili, & Tianzhu, 2008; Pal & Counts, 2011; Sakaki & Matsuo, 2010; Yamaguchi, Takahashi, Amagasa, & Kitagawa, 2010; Yu & Singh, 2000).
4 Twitter blogospheres
With the rapidly development of microblogging in recent years, the information diffusion in microblogging has received a considerable attentions from academic researchers. Microblogging such as Twitter and Sina Weibo (Chinese microblogging website) has rapidly developed as a recently emerging service due to its timeliness, convenience and it is a lightweight easy form of communication and share information. Posts or updates are made by succinctly describing one’s current status within a limit of 140 characters, known as “Tweets”. These posts are of research interest because they are where the social interactions are often played out. It must be stressed that there is strong evidence that people use them to find information (Costa & Sporns, 2006).
The Twitters’ ties are asymmetric, they are formed when a user follow someone, mostly because they are interested in topics that user publishes. Twitter allows a user to “follow” updates from other members who are added as “follower”. The “follower” concept, in Twitter perspective, represents the user who is following you. The “following” concept represents the user who you follow. Twitter user generally does not require permission to follow other users, and then it does not imply reciprocity. Twitter follower/following relationships resemble subscriptions to the RSS feeds of Websites more than friendship ties in Facebook. Twitter, therefore, constitutes a very open social network space, whose lack of barriers to access, e.g., even non-registered users are able to use Twitter to track breaking news on their chosen topics, from “World Economic Crisis” to “European Football Championship”. Twitter social networkers communicate with each other by posting tweets allowing for public interactive dialogue, if other users like or find its content truly interesting, they repost it or “Retweet” it. “Retweeting” is a key mechanism for information diffusion in microblogging. By allowing “Twitterers” to pass on information that they deem interesting, important, entertaining, etc, retweeting process behaves just like an informal recommendation system. Furthermore, when
someone “retweet” you, they are giving you a kind of reputation by sharing your post with their own followers or contacts. Users are more discerning when choosing what or who to retweet whereas not all tweets are reposted [48]. We selected Twitter as a case of study mostly because of retweet function. Starbird et al (Starbird, Palen, Hughes, & Vieweg, 2010) argue that people spread information that they feel or know to be newsworthy through retweeting. They also suggest that members of online communities use source credibility as a reputation to validate information. Source credibility refers to a message recipient’s perception of the credibility of an information source. It is defined as the extension to which an information source is perceived to be believable, competent, and trustworthy by information recipients (Gil & Artz, 2011).
5 Related works
Kwak and colleagues [48] rank Twitter users’ by popularity. The popularity was estimated by the number of followers, PageRank and by retweet count. The results show that all top users are either celebrities (actors, musicians, politicians, show hosts, and sports stars) or news media. They concluded that, only the number of followers does not reflect the influence of a user. The PageRank results showed again a list of celebrities on the top of the rank. The third rank approach (by retweet count) showed not only celebrities on the top but also news and media. From the results, we may conclude that the applied metrics were accomplished their aim, i.e., it evidenced the node popularity.
Cha et al (Cha et al., 2010) presented an empirical analysis of influence patterns in Twitter. They compared three different measures of influence: Indegree, retweets, and mentions. They examined how the three kinds of influential users performed in spreading popular news topics. The Indegree influence is the number of followers of a user, directly indicates the size of the audience for that user. Retweet influence, which indicates the ability of a user to generate content with pass-along value. Mention influence, which measures the number of mentions containing one’s name that indicates the ability of that user to engage others in a conversation. The authors found that, the most influential users were: news sources (CNN, New York Times), politicians (Barack Obama), athletes (Shaquille O’Neal), as well as celebrities (Ashton Kutcher, Britney Spears). The most retweeted users were content aggregation services (Mashable, TwitterTips, TweetMeme), businessmen (Guy Kawasaki), and news sites (The New York Times, The Onion). And finally, the most mentioned users were mostly celebrities.
Jianwei et al (Jianwei et al., 2008) proposed a new measure for characterizing the importance of a node with tunable parameters based on Degree centrality, Betweenness centrality and Closeness centrality. They used a Sexual Relation Network of the AIDS (SRNA) as a case of study. The authors argued that such measure has a stronger adaptability and are more discriminative when compared to other several centrality measures. It is important to mention, that such study has not clarified the research usability, the findings, and the meaning of “importance” of a node in the context of an AIDS network. In Twitter sphere, others measures are also used to rank node importance, such as follower count, co-follower rate (ratio between follower and following), frequency of tweets (updates), and measures of second level, for example, who your followers follow and so on (Bongwon, Lichan, Pirolli, & Chi, 2010; Boyd, Golder, & Lotan, 2010; L. Hong, Dan, & Davison, 2011; T. Hong, 2006; Weng, Lim, Jiang, & He, 2010; Yamaguchi et al., 2010).
6 Methodology
6.1 Network structure
In this subsection, we present the way to obtain the topological structure of social network based on retweet. Latest studies show that not only the network structural characteristics indicates the importance of a node, but also the user’s communication activity, i.e. the exchange of information for instance via messages, wall posts etc (Cha et al., 2010; Cheung & Lee, 2010; Shen, Syu, Nguyen, & Thai, 2012; Subbian & Melville, 2011). Therefore, we assume that “Retweeting” function is likely to be interpreted as a form of endorsement for both the message and the originating user (Weitzel, Quaresma, & de Oliveira, 2012). Retweet function represents the degree or level of interactions between users. By considering this feature, we proposed a network structure based on Retweet weighted ties named Retweet-Network or simply RT-network. Up to our knowledge, this is the first time that this topological network structure is modelled. We model the RT-network as a direct weighted graph GRT=(V,E,W)with the following properties:
• The set of nodes (denoting the set of users) V = {v1, v2,…}
• The set of edges (representing retweet function) E = {e1, e2,…}and
• If ∃ edge ek = (vi , vj) ∈ E, i.e., from vi to vj this means that user vi “retweet” user
• The set of weights (characterizing the strength of trust ties) W= {w1, w2,... } and
• The w(ek) is a function defined for edges as follows: λ + =(
∑
) ) ( total v k RT RT e w j Equation 1 Where the parameter∑
j
v
RT is the counted retweets for vj (target user) from a specific source user
vi and RTtotal is the total number of retweet of a target user. This fraction denotes how much make a
source user “trust” a particular target user. The parameter λ is a discount rate representing relationships (follower, following, friendship and no-relationship between source users and target users). Furthermore, λ intends to discount the weight of the follow phenomenon, since many celebrities and mass media have hundreds of thousands of followers; it defines smaller values to relationships that are “follower” or “following” (or both) and higher values when there is no relationship between users. Artists and celebrities attract a thousand of followers such as Lady Gaga, Britney Spears, Ashton Kutcher etc. For example, Ashton Kutcher is a classic Twitter celebrity phenomenon as an example of a star who has totally embraced the idea of Twitter celebrity-dom. In 2009, he became the first Twitter user to have more than one million followers and according to mid-2010, he had almost five million followers. Hence, the parameter λ is estimated according to the findings in Table 1, with regards to the relationships percentages. For our purposes, we set arbitrarily α according to the follower ratio: if vi is a follower of vj, thus the retweet is the event that is expected
to happen.
λ = 0.1 If a user vi is a follower of vj the parameter λ has lower weight,
λ = 0.9 in all other cases the parameter has higher weight.
6.2 Ranking reputation approach – RaR
Perhaps the most frequently used centrality measures are Degree (Dc), Closeness (Cc), Betweenness (Bc), and Eigenvector (Ec). Freeman (L. Freeman, 1979) proposed these three centrality measures. We used ORA software (http://www.casos.cs.cmu.edu) that computes such quantities and therefore let us briefly explain the main aspects of each one them. Bc is based on the shortest paths between nodes, focuses on the number of visits through the shortest path. In a directed graph, for a
vertex v, we denote the In-Degree Dcin (v) as the number of arcs arriving to v and the Out-degree Dcout (v) as the number of arcs starting from this node, thus Dc, of course is Dcin + Dcout . The Cc measures how close a vertex is to all other vertices in the graph. Ec was proposed by Bonacich (Bonacich, 1972), nodes with high values of Ec are linked to well-connected nodes and so may influence many others in the network either directly or indirectly through their connections. PageRank [61] is another common measure, it is generally used to rank WebPages and ultimately to rank “popularity”. Pagerank is a link analysis algorithm that assigns a numerical weight to each object of the information network, with the purpose of measuring its relative importance within the object set. It is defined as the stationary distribution of a stochastic process whose states are the nodes of the web graph, it computes the rank of websites by the number and quality of incoming links (Page, Brin, Motwani, & Winograd, 1999). In order to address the goal of this work and based on the outcomes above, we defined a new rank approach combining weighted centralities measures that best fit node importance. Thus, we defined the Rank Reputation -
j
v
RaR
(see Table 2 – set of results) as follows:n i i ij n i ij i v Max m m RaRj 1 1 } { = = ⋅ ⋅ =
∑
ω
ω
Equation 2 n j RaR n i i vj 1, 1, 1,..., 0 1 = = < <∑
= ω We model Rank Reputationj
v
RaR as a function of (M,ω) with the following properties: M = {m1,
m2,…} is a set of centrality measures, such as: Dc, Ec, Prank, Cc, Dcin, Dcout of the node vj, and ω =
{ω1, ω2,…} be a set of non-negative and normalized weights. Given the input directed weighted graph
)
W
,
E
,
V
(
G
RT=
as described herein, the RaRvjcomputes iteratively the reputation. In the first step, for each node vj it is calculated the metrics mi ∈ M. In second step, it is set out arbitrarily the weight ωi
∈ ω the estimated weights must follow the condition: , 1
1 = ∈ ∀
∑
= n i i i ω ω ω , where ωi = {0.0, 0.1,0.2,…1.0}, hence, it is possible that
∃
ω
k=
0
|
m
k*
ω
k=
0
. In the third step, for each node vj it iscomputed
∑
= ω n1
i mij i, thereafter compute RaRvj
. In the fourth step, we calculate the Map (
j v
RaR ) and
in the last step, return de best fit. The parameter Maxout refers to an Map = 100%, which means, the
best fit of user reputation.
Step1: Set out arbitrarily the weight ωi ∈ ω
1
,
n 1 i i i∈
ω
ω
=
ω
∀
∑
= Maxout = 100Step2: For each node vj compute
1
n 1 i i=
ω
∑
= Step3: Compute j vRaR
Step4: For each approach compute Map
Map is used to score document retrieval, it is an arithmetic mean average precision over a set of documents. As its name suggests, it averages precisions at individual ranks. In words, Map considers the precision at every relevant result in the list, and divides it by the result’s rank; then, the precision is averaged by dividing the sum of discounted precisions by the total number of relevant results. Which means, in optimal ranked retrieval system, a set of relevant retrieved documents are given by the top k
retrieved documents. Map is often used as an indicator for evaluating ranked retrieval results (Baeza-Yates & Ribeiro-Neto, 1999).
There are several sources of health information in Twitter blogosphere that people today reply on. We categorize them into following types of services: media, news, celebrities, public health agencies, public agencies, private health agencies and search engine. We considered “relevant documents”, in our case, “relevant users”, those that are only public health agencies since they have the ability to communicate information accurately. For example, if the site receives funding from commercial firms or private foundations, then the financial dependence has the potential to bias the information presented. For instance, if the purpose of the information is primarily to sell a product, there may be a conflict of interest since the manufacturer may not want to present findings that would discourage you from purchasing the product (Kumar, 2011).
In order to gain insight about our rank approach, we utilize the findings of (Cha et al., 2010; Jianwei et al., 2008; Kwak, Lee, Park, & Moon, 2010) as a baseline in our study (see Tabled 4).
6.3 Data sampling and scale free characteristics of the retweet network
We analyzed about 152 user profiles randomly chosen and their respective retweets. The data was acquired only from those who have some health interest, during March/April 2011. From these user accounts, we achieved 4350 retweet and 1232 user account and then we built the two datasets. The first dataset have the following fields: user screen name, class of user: target or source (target – “retweeted” user; source- user who “retweeted” you), number of tweets, number of followers, number of following, joined date and a short biography. The second dataset with the fields: source screen name, target screen name, kind of relationship between them (follower, following, friendship (both relationship), and no relationship), number of counted retweets (i.e., how many times target user is “retweeted” by a specific source user). The Table 1 provides an overview of datasets basic statistics. As can be seen, most of relationship is follower and a minority does not have relationship. Most of user main characteristics are individuals account and blogs respectively. It is about 8% are related to private healthcare, such as, medical association, non-governmental organization (NGO), and others non-profit organizations, and 17% are associated to public healthcare, i.e., the government department responsible for public health issues.
Relationships Follower = 64% Following = 15% Friendship = 14% None = 7% Retweets Mean = 3,4 Min = 1 Max= 527 Total = 4350
User account Total = 1232 users
Joined Date 2006 = 0,16% 2007 = 6,22% 2008 = 17,46% 2009 = 47,53% 2010 = 21,99% 2011 = 6,63% Main characteristic Individuals accounts ≈ 31% Blogs ≈ 26% Public Healthcare ≈17% News and media ≈ 13% Private healthcare ≈ 8%
Not classified or unknown class ≈ 2,2 % Celebrities ≈ 1%
Congresses and events ≈ 1%
Relevant users public healthcare = 212 ≈17% of the sample
We analyzed the scale free characteristics of networks. First of all, we check the histogram of degrees and we perform a direct non-linear Levenberg-Marquardt fit (Colizza, Pastor-Satorras, & Vespignani, 2007) by a power function
β
−α= x
x
f )
(
instead of to take the slope in the log-log plot. The reason here is very simple, we have only 1232 users and this method to determine parameters of power law would not be very precise due to many fluctuations on the tail of histogram with few points. It is important to mention that the most important point here is to corroborate that α < 3 in order to make stronger the power law behaviour (scale free characteristics) of the retweet network considered in our study. From this direct fit, which can be observed in Figure 1 (a) we obtain a very small exponent, α ≈ 1.66. This value corresponds to a power law with average and second moment (and therefore also its variance) that cannot be rigorously computed according to definition. The Figure 1 (b) corresponds to the same fit in log-log scale, i.e., it corresponds to the Figure 1(a) viewed in log-log scale. On the other hand, we estimate the power law exponent by a strict linear fit, estimating the slope in log-log scale that is a way intrinsically different than performing a non-linear fit. This linear fit is shown as Figure 1.Figure 1: Power law fits: (a) direct power law fit by Levemberg-Marquardt method (non-linear method) (b) The
same plot in log-log scale. The non-linear fit (in red) also was log-log scaled.
In this case we have a very small slope α ≈ 0.95. The fact is that the cloud of points sampled in the final of the plot should be masking the result. Therefore, we find a third opinion in a more laboured method: the method of moments. In this case, we compare experimental and theoretical factionary moments. The kth experimental moment of degrees is given by:
∑
==
n 1 i k ix
n
1
X
Equation 3where xi is the degree of of ith node and the kth theoretical moment is calculated by
)
1
k
(
)
1
(
dx
x
)
1
(
x
1 k k−
−
α
−
α
=
−
α
=
∫
∞ −α Equation 4So we should compare x and
x
k we compare the experimental ratios kkx
x
)
k
(
e
=
with theoretical ratios 1 k k k k ) 1 ( ) 1 k ( ) 2 ( x x ) k ( t − − α − − α − α = = Equation 5So we plot e(k) as function of k, with k from 0 to 1, with displacement Δk = 0.1 with plots of t(k) in the same figure for different α -values. In Figure 3 we show the experimental ratios (points) together with theoretical ratios (continuous lines) corresponding to different α-values. Numerically we determine α ≈ 2.37 as our best estimate. Therefore, we can conclude that α must be a number smaller
than 3 corroborating the hypothesis of power law distribution for the degrees. It is also important to mention that if on the one hand we observe different estimates to the exponents due to large fluctuations by considering a not so large number of points, on the other hand, all these estimates corroborate α < 3. This indicates heavy tail characteristics for the degree distribution in our retweet
network corroborating scale free network aspects.
7 Ranking results
The Table 2 illustrates the equations of the 10 best fits results according to the Equation 2 and the Rank reputation algorithm. We also compute
j
v
RaR
using the set of basic measures (Bc, Cc, Dc, Ec) in isolation. We found out that even when performed in isolation, the algorithm failed to achieve its goal, outcomes ( )j
v
RaR
Map ranged between 33% and 41%. The best fits are found for M5, M6 and M7
considering all records, i.e., all the 1232 nodes.
Measures RaR
Map
%M1 } { )) * 3 . 0 ( ) * 2 . 0 ( ) * 5 . 0 ( ) ( i i m Max Ec Dc Cc Bc ω + + + 54 M2 } { )) * 2 . 0 ( ) * 2 . 0 ( ) * 6 . 0 ( ) ( i i m Max Ec Dc Cc Bc ω + + + 56 M3 } { )) * 3 . 0 ( ) Pr * 1 . 0 ( ) * 6 . 0 ( ) ( i i m Max Dc ank Cc Bc ω + + + 56 M4 } { )) * 1 . 0 ( ) * 4 . 0 ( ) * 2 . 0 ( ) * 3 . 0 ( i i in out m Max Dc Cc Dc Cc ω + + + 56 M5 } { )) * 2 . 0 ( ) * 2 . 0 ( ) * 6 . 0 ( i i m Max Dc Ec Cc ω + + 58 M6 } { )) * 1 . 0 ( ) * 2 . 0 ( ) * 1 . 0 ( ) * 6 . 0 ( i i out in m Max Dc Dc Ec Cc ω + + + 58 M7 } { )) * 1 . 0 ( ) * 2 . 0 ( ) * 7 . 0 ( i i out in m Max Dc Dc Cc ω + + 58
Table 2: The set of equations – best fit
We also utilize the precision at k or p@k to evaluate the ranking reputation approach. The p@k is the proportion of relevant documents in the first k positions. This leads to measuring precision at fixed low levels of retrieved results, such as 10 or 30 documents (Baeza-Yates & Ribeiro-Neto, 1999). In our study we considered k = 212, representing the 212th position and total number of relevant documents.
Measures M1 M2 M3 M4 M5 M6 M7
p@k 50.94 54.25 49.53 49.53 54.72 55.19 56.13
Table 3: P@K at level of 212, and the level of 100% precision
As can be seen in Table 3, the maximum of p@212 was found in M7, approximately equal to 56%. By comparing the results obtained in two evaluated methods (Map and p@k), the best fit of these two methods are M7. In order to gain insights about our approach, we additionally employ two others
studies to carry out a comparative analysis. We use, the finding of: Kwak and colleagues research (Kwak et al., 2010) – the metrics are follower (M11), PageRank (M13) and retweet count (M12); and Jianwei et al (Jianwei et al., 2008) – the metric is a set of weighted measures (M14, M15). The findings of the comparative research were compiled Table 4.
Measures RaR Map %
M11 Follower 18 M12 Retweet count 39 M13 Pagerank 37 M14 (0.3* ) (0.1{* )}(0.6* )) i i m Max Dc Cc Bc ω + + 53 M15 (0.3* ) (0.2{* )}(0.5* )) i i m Max Dc Cc Bc ω + + 53
Table 4: The set of measures of the comparative analysis
8 Conclusions
This paper investigated the rich data structure of social media systems. We utilized the SNA method to figure out user’s reputation. In our study, reputation has the same meaning of reliable source of information in medical domains. We consider that, communication structure of Twitter is determined by two overlapping and interdependent networks – one based on follower-following relationships, and other relatively short-term and emergent, based on shared interest in a topic or event, often coordinated by a retweet function. Therefore, Retweet Network must be understood as a separated part from follower/following Network. We found out some interesting results. The majority of Twitter accounts are individual or blogs, since this is the proper nature of Web 2.0. Web 2.0 is driven by participation and collaboration among users, most obviously apparent in social networking, social bookmarking, blogging, wikis etc.
We also found out that Reputation Rank Approach is responsive to Dcin and Dcout. These metrics
are present in all best-performing results of Network. By contrast, the average precision RT-Network worst-performing results are those that use the PageRank and Bc. All metrics in isolation failed in reaching user reputation, specially the Cc and follower count metrics. Almost all measure of
j
v
RaR achieved about 90% of p@10 performance measure. The third rank approach (by retweet count in Table 4) showed not only celebrities on the top but also news and media. From the results, we may conclude that the applied metrics were accomplished their aim, i.e., it evidenced the “node popularity. ”
The study gives us a clear understanding of the how measure selection can affect the reputation rank (especially in medical domain). Choose the most appropriate measure depends on that we want to represent. The PageRank operate look alike “edges counts” as the “popularity” measures. We noticed that popularity (or key position in a graph) does not necessarily refer to reputation. The Bc metric is an important quantity to characterize how influential a node (user) is in communications between each pair of vertices, it represents a “gatekeeper” between groups node, and yet both metrics failed. By contrast, the Cc and Dcin metrics fulfilled the rank goal, i.e., in expressing the reputation. The major
contributions of this work were mostly providing a new methodology to rank trustworthy source using a new network structure based on retweet ties. Our methodology to compute rank and the way to build
the network topology seem to supply an interesting online method to determine user reputation in the medical domain. Moreover, we also verified that in Twitter community, trust plays an important role in spreading information; the culture of “retweeting” seems to have a good potential to reach trust. For future work we will conduct a study to evaluate other social media that uses similar mechanism, based on endorsement approach. We verified that trust plays an important role in spreading credible information.
References
Ahn, Y.-Y., Han, S., Kwak, H., Moon, S., & Jeong, H. (2007). Analysis of topological characteristics of huge
online social networking services. Paper presented at the INTERNATIONAL CONFERENCE ON
WORLD WIDE WEB, Banff, Alberta, Canada.
Anderson, J. G. (2004). Consumers of e-Health: Patterns of Use and Barriers. Social Science Computer Review,
22(2), 242-248. doi: 10.1177/0894439303262671
Antheunis, M. L., Valkenburg, P. M., & Peter, J. (2010). Getting acquainted through social network sites: Testing a model of online uncertainty reduction and social attraction. Computers in Human Behavior, 26(1), 100-109.
Baeza-Yates, R. A., & Ribeiro-Neto, B. (1999). Modern Information Retrieval: Addison-Wesley Longman Publishing Co., Inc.
Balkundi, P., & Kilduff, M. (2005). The ties that lead: A social network approach to leadership. The Leadership
Quarterly, 16(6), 941-961. doi: 10.1016/j.leaqua.2005.09.004
Barabási, A.-L. (2003). Linked : How everything is connected to everything else and what it means for business,
science, and everyday life. New York: Plume.
Bernstam, E. V., Sagaram, S., Walji, M., Johnson, C. W., & Meric-Bernstam, F. (2005). Usability of quality measures for online health information: Can commonly used technical quality criteria be reliably assessed? International Journal of Medical Informatics, 74(7–8), 675-683. doi:
10.1016/j.ijmedinf.2005.02.002
Boberg, E. W., Gustafson, D. H., Hawkins, R. P., Offord, K. P., Koch, C., Wen, K.-Y., . . . Salner, A. (2003). Assessing the unmet information, support and care delivery needs of men with prostate cancer. Patient
Education and Counseling, 49(3), 233-242. doi: 10.1016/s0738-3991(02)00183-0
Bonacich, P. (1972). Technique for analyzing overlapping memberships. Sociological Methodology, 4, 176-185. doi: citeulike-article-id:1036885
Bongwon, S., Lichan, H., Pirolli, P., & Chi, E. H. (2010). Want to be Retweeted? Large Scale Analytics on
Factors Impacting Retweet in Twitter Network.
Borgatti, S. P., Mehra, A., Brass, D. J., & Labianca, G. (2009). Network Analysis in the Social Sciences. Science,
323(5916), 892-895.
Boyd, D., Golder, S., & Lotan, G. (2010). Tweet, Tweet, Retweet: Conversational Aspects of Retweeting on
Twitter, Los Alamitos, CA, USA.
Bruns, A. (2009). Social Media: Tools for User-Generated Content Social Drivers behind Growing Consumer Participation in User-Led Content Generationts only (Vol. 2, pp. 47-47). Australia: Smart Services CRC Pty Ltd.
Cai, D., Shao, Z., He, X., Yan, X., & Han, J. (2005). Community mining from multi-relational networks.
Knowledge Discovery in Databases: PKDD 2005, 445-452.
Cha, M., Haddadi, H., Benevenuto, F., & Gummadi, K. (2010). Measuring User Influence in Twitter: The Million
Follower Fallacy. Paper presented at the Proceedings of the 4th International AAAI Conference on
Weblogs and Social Media (ICWSM-2010), Washington, D.C.
Cheung, C. M. K., & Lee, M. K. O. (2010). A theoretical model of intentional social action in online social networks. Decision Support Systems, 49(1), 24-30. doi: 10.1016/j.dss.2009.12.006
Colizza, V., Pastor-Satorras, R., & Vespignani, A. (2007). Reaction–diffusion processes and metapopulation models in heterogeneous networks. Nature Physics, 3(4), 276-282. doi: 10.1038/nphys560
Cosley, D., Huttenlocher, D., Kleinberg, J., Lan, X., & Suri, S. (2010). Sequential Influence Models in Social Networks. from http://www.aaai.org/ocs/index.php/ICWSM/ICWSM10/paper/view/1530/1829
Costa, L. d. F., & Sporns, O. (2006). Diversity of Cortical States at Non-Equilibrium Simulated by the Ferromagnetic Ising Model Under Metropolis Dynamics. 18. arXiv preprint cond-mat/0604089
Crandall, D. J., Backstrom, L., Cosley, D., Suri, S., Huttenlocher, D., & Kleinberg, J. (2010). Inferring social ties from geographic coincidences. Proceedings of the National Academy of Sciences, 107, 22436-22441. Dana, J., & Loewenstein, G. (2003). A social science perspective on gifts to physicians from industry. JAMA,
290, 252-255.
de Boer, M. J., Versteegen, G. J., & van Wijhe, M. (2007). Patients’ use of the Internet for pain-related medical information. Patient Education and Counseling, 68(1), 86-97. doi: 10.1016/j.pec.2007.05.012
Dorogovtsev, S. N., & Mendes, J. F. F. (2003). Evolution of networks : from biological nets to the Internet and
WWW. Oxford [u.a.]: Oxford University Press.
Easley, D., & Kleinberg, J. (2010). Networks, Crowds, and Markets ; Reasoning about a Highly Connected
World. [S.l.]: Cambridge University Press.
Eysenbach, G. (2002). Empirical Studies Assessing the Quality of Health Information for Consumers on the World Wide Web: A Systematic Review. JAMA: The Journal of the American Medical Association,
287, 2691-2700.
Freeman, L. (1979). Centrality in social networks: Conceptual clarification. Social Networks, 1(3), 215-239. doi: citeulike-article-id:278955
doi: 10.1016/0378-8733(78)90021-7
Freeman, L. C. (1996, 1996). Cliques, Galois lattices, and the structure of human social groups. Social Networks,
18, 173-187.
Gil, Y., & Artz, D. (2011). Towards Content Trust of Web Resources. Journal of Web Semantics: Preprint
Server, 5(4).
Granovetter, M. S. (1973). The Strength of Weak Ties. American Journal of Sociology, 78(6), 1360-1380. Griffiths, M. K., Tang, T. T., Hawking, D., & Christensen, H. (2005). Automated Assessment of the Quality of
Depression Websites. J Med Internet Res, 7(5), e59.
Hong, L., Dan, O., & Davison, B. D. (2011). Predicting popular messages in Twitter. Paper presented at the INTERNATIONAL CONFERENCE ON WORLD WIDE WEB, Hyderabad, India.
Hong, T. (2006). The influence of structural and message features on Web site credibility. Journal of the
American Society for Information Science and Technology, 57(1), 114-127. doi: 10.1002/asi.20258
Hu, Y., & Shyam Sundar, S. (2009). Effects of Online Health Sources on Credibility and Behavioral Intentions.
Communication Research, 37(1), 105-132. doi: 10.1177/0093650209351512
Hughes, B., Joshi, I., Lemonde, H., & Wareham, J. (2009). Junior physician's use of Web 2.0 for information seeking and medical education: a qualitative study. Int J Med Inform, 78(10), 645-655. doi: 10.1016/j.ijmedinf.2009.04.008
Ibarra, H. (1992). Homophily and Differential Returns: Sex Differences in Network Structure and Access in an Advertising Firm. Administrative Science Quarterly, 37(3), 422-422.
Jianwei, W., Lili, R., & Tianzhu, G. (2008). A New Measure of Node Importance in Complex Networks with
Tunable Parameters.
Jøsang, A., Ismail, R., & Boyd, C. (2007). A survey of trust and reputation systems for online service provision.
Decision Support Systems, 43(2), 618-644. doi: 10.1016/j.dss.2005.05.019
Kochanek, K. D., Xu, J. Q., & Murphy, S. L. (2011). Deaths: Preliminary Data for 2009 (Vol. 59). Hyattsville, United States: National Center for Health Statistics.
Kumar, V. (2011). Impact of Health Information Systems on Organizational Health Communication and Behavior. Internet Journal of Allied Health Sciences and Practice, V. 9, n. 2.
Kwak, H., Lee, C., Park, H., & Moon, S. (2010). What is Twitter, a social network or a news media?, Raleigh, North Carolina, USA.
McPherson, M., Smith-Lovin, L., & Cook, J. M. (2001). Birds of a feather: Homophily in social networks. Annual
review of sociology, 27, 415-444.
Page, L., Brin, S., Motwani, R., & Winograd, T. (1999). The PageRank Citation Ranking: Bringing Order to the Web: Stanford InfoLab.
Pal, A., & Counts, S. (2011). Identifying topical authorities in microblogs, Hong Kong, China.
Sabater, J., & Sierra, C. (2005). Review on Computational Trust and Reputation Models. Artif. Intell. Rev., 24(1), 33-60. doi: 10.1007/s10462-004-0041-5
Sakaki, T., & Matsuo, Y. (2010). How to Become Famous in the Microblog World. from http://www.aaai.org/ocs/index.php/ICWSM/ICWSM10/paper/view/1505/1887
Shalizi, C. R., & Thomas, A. C. (2011). Homophily and Contagion Are Generically Confounded in Observational Social Network Studies. Sociological Methods & Research, 40, 211-239.
Shen, Y., Syu, Y. S., Nguyen, D. T., & Thai, M. T. (2012). Maximizing circle of trust in online social networks. Starbird, K., Palen, L., Hughes, A. L., & Vieweg, S. (2010). Chatter on the red: what hazards threat reveals
about the social life of microblogged information. Paper presented at the Proceedings of the 2010 ACM
conference on Computer supported cooperative work, Savannah, Georgia, USA.
Stvilia, B., Mon, L., & Yi, Y. J. (2009). A model for online consumer health information quality. Journal of the
American Society for Information Science and Technology, 60(9), 1781-1791.
Subbian, K., & Melville, P. (2011, 9-11 Oct. 2011). Supervised Rank Aggregation for Predicting Influencers in
Twitter. Paper presented at the Proceedings of the IEEE International Conference on Privacy, Security,
Risk, and Trust, and Third international Conference on Social Computing (SOCIALCOM), Boston, Massachusetts.
Wang, Y., & Liu, Z. (2007). Automatic detecting indicators for quality of health information on the Web.
International Journal of Medical Informatics, 76(8), 575-582. doi: 10.1016/j.ijmedinf.2006.04.001
Wasserman, S. (1999). Social network analysis : methods and applications. Cambridge: Cambridge University Press.
Watts, D. J. (2004). The “New” Science of Networks. Annu. Rev. Sociol., 30(1), 243-270. doi: 10.1146/annurev.soc.30.020404.104342</p>
Watts, D. J., & Strogatz, S. H. (1998). Collective dynamics of "small-world" networks. Nature, 393(6684), 440-442.
Weaver, J. B., Thompson, N. J., Weaver, S. S., & Hopkins, G. L. (2009). Healthcare non-adherence decisions and internet health information. Computers in Human Behavior, 25(6), 1373-1380. doi: 10.1016/j.chb.2009.05.011
Weitzel, L., Quaresma, P., & de Oliveira, J. P. M. (2012, 26-29 March 2012). Evaluating Quality of Health
Information Sources. Paper presented at the Advanced Information Networking and Applications
(AINA), 2012 IEEE 26th International Conference on.
Weng, J., Lim, E.-P., Jiang, J., & He, Q. (2010). TwitterRank: finding topic-sensitive influential twitterers, New York, New York, USA.
Yamaguchi, Y., Takahashi, T., Amagasa, T., & Kitagawa, H. (2010). TURank: Twitter User Ranking Based on User-Tweet Graph Analysis. In L. Chen, P. Triantafillou & T. Suel (Eds.), Web Information Systems
Engineering – WISE 2010 (Vol. 6488, pp. 240-253): Springer Berlin / Heidelberg.
Yu, B., & Singh, M. P. (2000). A Social Mechanism of Reputation Management in Electronic Communities. In M. Klusch & L. Kerschberg (Eds.), Cooperative Information Agents IV - The Future of Information
