• No results found

Reputation system design is the. Advanced Feedback Management for Internet Auction Reputation Systems. Trust & Reputation Management

N/A
N/A
Protected

Academic year: 2021

Share "Reputation system design is the. Advanced Feedback Management for Internet Auction Reputation Systems. Trust & Reputation Management"

Copied!
7
0
0

Loading.... (view fulltext now)

Full text

(1)

Tr

us

t

&

R

ep

ut

at

io

n

M

an

ag

em

en

R

eputation system design is the most important thing in most social platforms. With Internet auction systems, especially — where users invest great amounts of money — it’s crucial that those users get as much information as possible to improve the reliability of transactions. Reputation and trust management systems are based primarily on user feedback, but most platforms use old-fashioned com-ment counting and a [–1, 0, 1] scale for comment valuation. However, such situations don’t encourage sellers to improve their services.

To address this problem, buyers should be able to explicitly express their detailed opinions about an auc-tion and the seller. The Detailed Seller Report (DSR) eBay introduced in 2009 is still controversial (in part because

the algorithm causes problems for low-volume sellers), but this approach moves one step closer toward extend-ing the simple user-feedback interface. A well-designed feedback system and better-quality feedback can influence users’ trust toward the system itself.

Here, we extend auction site feed-back mechanisms by extracting infor-mation embedded in user comments, assigning this feedback value for use in reputation systems, and presenting the computed data to the user. To improve feedback, we suggest encouraging users to fill out a simple inquiry form at the end of an auction. This inquiry’s subject should be adapted to take into account the item category, the feedback type, and the comments’ semantic meaning.

Recent research shows that for posi-tive comments, the added information is

Electronic auction reputation systems have improved in recent years. However, most don’t rely on user feedback but are still bound to old-fashioned comment counting while substantial information embedded in those comments is omitted. The authors’ system manages and learns from user feedback and considers auctions’ context, possible types of complaints, and the structure of connections between those complaints. They propose amplifying a reputation system algorithm to estimate the reported complaints’ harmfulness. Their results are based on a real-world dataset from a leading Eastern European online auction provider.

Tomasz Kaszuba Polish-Japanese Institute of Information Technology Albert Hupa University of Warsaw Adam Wierzbicki Polish-Japanese Institute of Information Technology

Advanced Feedback

Management for Internet

Auction Reputation Systems

(2)

negligible.1 So, in our system, we consider only information from negative and neutral comments.

Complaints Model

Trust toward sellers is crucial for Internet auction platforms. Some trust enforcement mechanisms exist, such as DSR or escrow (an insurance system for auctions), but these don’t build trust among a site’s users. The auction platform provider should build trust in a bottom-up manner, starting with buyers, who express their feelings after the auc-tion. The best possible way to improve trust is to ask buyers their opinions about sellers. Currently, eBay asks buyers the same four permanent ques-tions after every auction, then computes the DSR value, which influences the sellers’ fees, pro-motions, the order of search, and so on. As our research shows, users can express much more subtle opinions about sellers than these ques-tions might indicate. In our opinion, classifica-tion and understanding of users’ complaints can help build a better trust-management system.

Additionally, existing reputation systems don’t distinguish between different kinds of negative or neutral user feedback. They also use a very simple reputation algorithm. As long as we treat all negative feedback equally, we can’t distinguish a purposeful behavior from an acci-dental one. For example, a great difference exists between sending the wrong color or size of T-shirt to a user and not sending the T-shirt at all.

To address these issues, we developed a model in which we can classify complaints against sellers and thus provide much more information about their inappropriate behavior.

The Dataset

To create our classification model, we obtained a real-world dataset from Allegro (www.allegro. pl), the leading Eastern European online auc-tion provider. In this service, each aucauc-tion has an explicit deadline, and all current bids are exposed to all participants. In most auctions, bidders can specify the maximum price they want to pay for an item, and Allegro’s proxy bid system automatically raises the bid, using only the minimum amount necessary for the bidder to maintain the top position. Bidders can also increase their maximum price at any moment. When the auction terminates, the bidder with the highest bid wins. Allegro also has various multi-item (“Buy now!”) auctions in which sell-ers can sell more than one item (leading to more

than one winner). In such auctions, every bid is a winning bid until no more items are available.

For our model, we selected a subset of 9,500 sellers and buyers and their 285,000 auctions listed in 16,000 categories. We performed our research on a subset (15,000) of negative and neutral comments from 1.7 million total positive and nonpositive comments. The unequal amount of auction feedback versus the number of auc-tions is due to the multi-item type aucauc-tions.

We mined the information from users’ com-ments using two independent classification approaches: top down and bottom up. These approaches helped us compare the outcomes — that is, the different types of complaints — on the basis of which we created a taxonomy by connect-ing those types accordconnect-ing to different meanconnect-ings.

Classification Methods

The top-down approach we used to develop our taxonomy involved creating a simple typology tree that employed a semiautomatic method using regular expressions. We designed and implemented the regex creator tool, which lets us create new regular expressions and assign the patterns to complaint types. In the bottom-up approach, we used advanced data mining techniques to cluster the co-occurring words into groups. We applied the Newman-Girvan algorithm for community detection.2,3 This approach is based on the measures of short-est path and betweenness centrality calculated for edges. Applying this algorithm generated sets of words that usually occurred together in our dataset. We treated these sets as meaning-ful types of complaints. Then, we examined the results from both methods and created the tree structures Figure 1 presents. More details about this work are available in our previous work.4

Taxonomy of Complaints

Within the taxonomy Figure 1 shows, we dis-tinguish two kinds of losses due to fraud: time-related and money-related. We marked complaints related to lost time with striped lines, whereas those in light gray are related to mon-etary loss. In addition, we marked the four crite-ria that match those in eBay’s DSR (the question about communication with the seller covers two of the categories in our taxonomy). We observed two general groups of complaints: seller-behav-ior-related and item-related. In the first group, we include the following seller behavior:

(3)

Fraudulent behavior includes shill bidding (that is, when a third party, often con-nected to the seller, raises the auction price intentionally) or shipping overcharges. We considered only explicitly formulated accu-sations, not those computed from historical auction data.

No response complaints mean communi-cating with the seller after the auction was impossible. The seller didn’t answer phone calls or respond to emails.

Odd behavior involves the seller behaving in a completely unpredictable manner. For instance, communication with the seller was possible but handicapped, the seller sent the item with a delay, or he or she didn’t define the payment method and shipping price. The second group of complaints is related strictly to the item:

Item not sent or lost means the buyer never received the item, either because it was never sent, or because, as sellers sometimes argue, the courier or post office lost it. • No product to sell occurs when the seller

claims that the item was already sold to another buyer or that the item is no lon-ger for sale. In this case, the buyer doesn’t receive the item.

Careless packaging indicates that the seller

didn’t take care in packaging the items and includes situations in which the received item arrived damaged. Verifying whether the seller sent a damaged item or the item was destroyed during shipment isn’t possible. • Wrong item means the seller sent the wrong

item (or wrong color or type), or the received item wasn’t complete.

Item not as expected indicates that the item seems to be illegal goods (a fake, or pirated software) or just doesn’t satisfy the buyer. Because the groups are disjoint, we can con-nect every comment to one or more of them. This provides more information about the seller’s pro-file, which is crucial for sellers. The category tree covers only the meaningful accusations against the seller. We don’t distinguish the most offen-sive comments (swearing and threats) unless we find at least one complaint from the list.

Complaint Classification

Using regular expressions prepared via our two classification methods, we partitioned all negative and neutral feedback from our data-set into the detailed types represented in the complaint taxonomy. Each complaint type has its own meaning and also a unique set of regu-lar expression patterns. In our evaluation, we used the types classified into the general level only to obtain more legible results. We group

*

*

*

*

*

No payment info Not as described No product to sell Item not received Item received Item related User related Seller

*

Money related Time related Main group DSR criteria Not sent or lost Item not as expected Item wrong Careless packaging Odd behavior No response Fraudulent behavior Low quality Fake or illegal Item not complete Item different Item damaged Poorly packaged Overcharge shipping Shill bidding Item sent late Comm. misbehaving General Detailed

Figure 1. Typology of complaints against sellers. We can discriminate between user- and item-related problems. Accusations against seller behavior are mainly time-related, whereas item-related accusations are connected to a loss of money.

(4)

the patterns from the detailed level along the category tree.

We tested all negative and neutral comments buyers made and assigned each to types in our taxonomy. We matched each comment against all patterns from our model. A comment could be assigned to more than one pattern from dif-ferent types. In Figure 2, we present normal-ized results for all neutral or negative feedback separately. In addition, we present the percent-age results jointly for all nonpositive feedback (negative or neutral comments).

Our regular expression tool matched 68 per-cent of the negative comments and 54 perper-cent of the neutral comments for sellers. The differ-ence in classification quality between negative and neutral feedback occurs because neutral comments contain fewer complaints, which pro-vide the most useful information.

Unclassified comments contained mostly useless information (no specified reason or lots of spelling errors). We reduce the amount of such feedback by enabling users to choose one of our proposed complaint types from a list instead of editing comments themselves. How-ever, users retain the ability to edit comments afterward to add more information if desired.

Negative feedback. Figure 2 shows how fre-quently complaints occurred against sellers in our dataset. Most negatives are due to a lack of response from the seller or to buyers not receiv-ing the item. This is predictable because users don’t like to be uninformed, especially when they risk their money. A significant amount of

negative feedback is due to problems with the item, such as the seller sending the wrong or a low-quality item. A few buyers made direct accusations of shill bidding or excess shipping costs. We also noticed some situations in which the seller refused to sell the item and informed the buyer about it.

Neutral feedback. Buyers gave neutral feedback in most cases when the item didn’t live up to their expectations or was different (for exam-ple, a different color or size) than described in the auction. Seller behavior, such as a buyers’ problems understanding the seller or delays in sending the item, were also frequent reasons for neutral, rather than negative, feedback. Com-pared with negative feedback, we can observe a significant drop (almost 50 percent) in com-plaints related to not sending an item or ignor-ing the buyer.

Complaints Grading

To make our research more applicable to trust or reputation systems in general, we propose a simple method for rating the types of com-plaints according to their harmfulness. We based this method on the percentage points of negative and neutral comments in each com-plaint category. To verify our method and detect independent rules of grading, we con-ducted an opinion poll among real Internet auction users. We juxtaposed the results from both methods in Figure 3. All values for the computational method were generated from nonpositive feedback (negative or neutral). Less Negative Neutral Nonpositive Item wrong Item not as expected Careless packaging Odd behavior

Item not sent or lost No product to sell No reponse Fraudulent behavior Fr equency of occurr ence 35 30 25 20 15 10 5 0

Figure 2. Results for seller complaints. Buyers make neutral comments more often when there is a problem with the item or its state, whereas they make negative comments when the seller doesn’t follow the rules (communication problems or lack of item).

(5)

harmful complaint types have lower values in the figure.

Computational Method

We define harmfulness as the balance between the frequency of occurrence of negative and neutral feedback (percentage points). We pute the harmfulness for every type in our com-plaint taxonomy. A type of comcom-plaint tends to be more harmful if we’ve classified more negative than neutral feedback into that type. According to this computational method, the most harm-ful seller behaviors are lack of response and not sending the item after receiving payment. Users are more forgiving with cases connected to the item’s condition. Shipment delays and mistakes are more often graded as neutral than negative.

Opinion Poll

To verify our approach’s correctness, we con-ducted an opinion poll among real Internet auction users from sites such as Allegro and eBay. We received 208 answers from people between the ages of 19 and 59. One hundred and forty-eight declared that they sold goods on auction systems. We asked respondents for their subjective opinions about each catego-ry’s harmfulness.

Results obtained from the opinion poll show that more frequent types of complaints aren’t necessarily considered more harmful by respondents. Users seem to be more tolerant to a lack of response from the seller or to when sell-ers declare after the auction that they no longer have the item to sell. According to our

respon-dents, the most harmful behaviors regard the item’s condition, as when sellers send a dam-aged, incomplete, or different item.

Improving the Feedback System

As mentioned previously, eBay introduced its DSR system to let buyers rate sellers on four different criteria: product description accu-racy, communication between the seller and buyer, shipping time, and shipping charge. Buy-ers can leave one to five stars for each criteria. Although the system design is convenient for buyers, eBay doesn’t instruct them on how to use it or reveal the reasons behind its criteria.

Compared to our model, criteria used in DSR cover most user-related problems but don’t take into consideration item-related complaints (only the question about the item description falls into this category). Moreover, DSR uses iden-tical criteria for every type of category and value of an item, which can be a serious con-straint in letting users express themselves. Our approach covers all complaints in greater detail with respect to the buyers’ harmfulness scale. We suggest adapting such questions to auction feedback systems, which should utilize user-provided information to create a seller profile, thereby influencing the reputation system.

Complaint Co-Occurrence

Our research shows that certain groups of com-plaints often occur together. Such connections can improve the description of the seller’s pro-file when we can’t presume it from a simple classification. To detect such clusters, we cre-No reponse Item not sent

or lost No product to sell Fraudulent behavior Careless packaging Item not as expected Odd behavior Item wrong Harmfulness 9 8 7 6 5 4 3 2 1 0 Computational Opinion poll 8 3 7 8 6 4 5 1 4 7 3 6 2 5 1 2

Figure 3. Harmfulness grading. According to the opinion poll, problems with the item are the most harmful (that is, have the highest value). The computational method’s results show that the sellers’ behavior is also important.

(6)

ated symmetric networks in which vertices rep-resent types of complaints and edges stand for their co-occurrence. We used the frequency of co-occurences as weights for a Newman-Girvan algorithm.2,3 We ran the algorithm twice to locate subgroups using the general and detailed groups from our complaint model (see Figure 1).

General Groups

We applied the Newman-Girvan clustering algorithm and generated a similarity tree for eight complaints. Figure 4a presents the results for the general groups. As we can see, two main groups occurred separately. One is related to an item’s condition, which corresponds to the results from the opinion poll (users evaluated the three types in this group as more harmful than did our computational scale). These com-plaints frequently occur with accusations of shill bidding or shipping overcharges.

The second group is connected with the seller’s behavior, such as no response, not sending or delay in sending an item, or no payment information.

Detailed Groups

We ran the algorithm again on the detailed groups (15 complaints) to detect more precise results. Figure 4b shows this similarity tree. We can spot two groups of co-occurring com-plaints. At the bottom, we can see a group con-nected strictly with item descriptions. The cluster includes situations such as sending an incom-plete or different item, items with a poor descrip-tion, or even the lack of the item itself. At the higher (less-connected) level, we see misbehavior in communication or overcharge shipping, which can be characteristics of fraudulent sellers.

The second strong cluster is related to late shipments and a lack of response from the seller, which is typical for sellers who aren’t interested in communication with the buyer.

W

e suggest using our solutions, including complaints grading and complaints co-occurrence, together to improve existing repu-tation systems. Our improvements are based on user-expressed feedback, so they’re applicable to most auction platforms. Sites can adapt these improvements to their current reputation sys-tems with little effort (they require recomputing the frequencies of occurrence and harmfulness).

Internet auction platforms can also use our rating schemes to detect and fight against fraud and thereby gain more trust from users. Rating complaints can be a good alternative to the con-troversial DSR and can be easily implemented into auction platforms. Using types of complaints for the seller profile description can improve the service, and historical data about the seller’s pro-file type can be available instead of or in con-junction with a list of comments. The seller’s profile type can also affect search visibility or seller fee discounts. Auction services can use co-occurrence between the types of complaints as a building block for the adaptive questionnaire system. We suggest applying such a system to the feedback module in reputation management systems (instead of using an immutable list, as in eBay-like systems). Aggregated feedback or sta-tistics can also be available to sellers in the dash-board or similar seller-support tool, so they can improve their services based on user comments.

In the future, we plan to integrate our model with the ProtoTrust tool,5 an interactive Web browser extension that helps users in the deci-sion-making process using trust management No_reponse Item_not_sent_or_ lost No_product_to_sell Fraudulent_behavior Careless_packaging Item_not_as_expected Odd_behavior Item_wrong Poorly_packaged Item_low_quality Communication_misbehaving Item_not_sent_or_ lost No_product_to_sell Item_sent_late Item_not_sent_or_ lost Item_illegal_or_fake Item_damaged Item_wrong Overcharge_shipping No_product_to_sell Item_not_as_described Item_not_complete Item_different (a) (b)

Figure 4. Similarity tree of complaints. We applied a clustering algorithm to determine the co-occurrence of types of complaints for (a) general groups and (b) detailed groups from our

(7)

techniques. ProtoTrust can compute more com-plex measures than simple reputation (risk, prob-ability of fraud, and average selling price, for example) and takes into consideration an auc-tion’s context. By integrating with ProtoTrust, we hope to create a helpful, user-friendly tool that lets users detect unreliable contractors.

Acknowledgments

The Polish Ministry of Science and Higher Education funded the work reported in this article under research grant N N516 4307 33.

References

1. A. Wawer and R. Nielek, “Remove Positive Comment Descriptions between Informativity and Sentiment,”

Proc. Associated Workshops and Doctoral Consortium of the 13th East European Conf. (ADBIS 09), LNCS 5968, Springer, 2009.

2. M. Girvan and M.E.J. Newman, “Community Structure in Social and Biological Networks,” Proc. Nat’l Acad-emy of Sciences, vol. 99, no. 12, 2002, pp. 7821–7826. 3. M.E.J. Newman and M. Girvan, “Finding and

Evaluat-ing Community Structure in Networks,” Physical Rev.,

vol. E 69, no. 026113, 2004.

4. T. Kaszuba, A. Hupa, and A. Wierzbicki, “Comment Classification for Internet Auction Platforms,” Local Proc. 13th East-European Conf. (ADBIS 09), JUMI Pub-lishing House, 2009, pp. 374–384.

5. T. Kaszuba et al., “ProtoTrust: An Environment for Improved Trust Management in Internet Auctions,”

Local Proc. 13th East-European Conf. (ADBIS 09), JUMI Publishing House, 2009, pp. 385–398.

Tomasz Kaszuba is a fourth-year PhD student and assistant at the Polish-Japanese Institute of Information Technology (PJIIT). He has a master’s degree in distributed comput-ing from PJIIT. Contact him at kaszubat@pjwstk.edu.pl. Albert Hupa is a sociologist at the University of Warsaw.

He has a PhD from the Institute of Applied Social Sci-ences at the University of Warsaw. Contact him at albert.hupa@gmail.com.

Adam Wierzbicki is an assistant professor at the Polish-Jap-anese Institute of Information Technology. He has a PhD in Internet communications from Warsaw University of Technology. Contact him at adamw@pjwstk.edu.pl.

Related Work in Electronic Auction Fraud Detection

M

ost recent work in electronic auction fraud detection has focused only on a seller’s profile.1,2 Researchers

have devoted considerable work to inducing users to behave properly,2,3 and to detecting fraudulent users.4,5 Recently, Yatel

Yang and his colleagues designed a system for detecting dishon-est ratings.6 Their system complements the one we describe

in the main text because it can filter out unfair complaints. Some tools are dedicated to detecting fraudulent sellers7 or

entire cliques of fraudulent agents (such as NetProbe8). Bezael

Gavish and Christopher L. Tucci presented sellers’ swindling methods in Internet auctions,9 whereas Dawn C. Gregg and

Judy E. Scott have proposed a model of complaints against sell-ers,1 although they haven’t discussed a model against buyers.

While their model is similar to ours, they use a manual pro-cess to classify feedback and don’t employ bottom-up mining techniques to detect the complaint groups. They also haven’t proposed a way to improve the grading of feedback types. Although we don’t discuss it in this article, our recent work has created a model of complaints against buyers.10

References

1. D.G. Gregg and J.E. Scott, “A Typology of Complaints about eBay Sellers,”

Comm. ACM, vol. 51, no. 4, 2008, pp. 69–74.

2. P. Resnick and R. Zeckhauser, “Trust among Strangers in Internet Transac-tions: Empirical Analysis of eBay’s Reputation System,” The Economics of the

Internet and E-Commerce, Advances in Applied Microeconomics series, M.R.

Baye, ed., Elsevier Science, 2002, pp. 127–157; www.si.umich.edu/ presnick/ papers/ebayNBER/index.html.

3. C. Dellarocas, “Immunizing Online Reputation Reporting Systems against Unfair Ratings and Discriminatory Behavior,” Proc. 2nd ACM Conf. Electronic

Commerce (EC 00), ACM Press, 2000, pp. 150–157.

4. D.H. Chau and C. Faloutsos, “Fraud Detection in Electronic Auction,” European Web Mining Forum, Proc. European Conf. Machine Learning and

Principles and Practice of Knowledge Discovery in Databases, 2005; http://cite

seerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.88.2438.

5. S. Rubin et al., “An Auctioning Reputation System based on Anomaly Detec-tion,” Proc. 12th ACM Conf. Computer and Communications Security (CCS 05), ACM Press, 2005, pp. 270–279.

6. Y. Yang et al., “Defending Online Reputation Systems against Collaborative Unfair Raters through Signal Modeling and Trust,” Proc. 2009 ACM Symp.

Applied Computing, ACM Press, 2009, pp. 1308–1315.

7. T. Kaszuba et al., “ProtoTrust: An Environment for Improved Trust Manage-ment in Internet Auctions,” Local Proc. 13th East-European Conf. (ADBIS 09), JUMI Publishing House, 2009, pp. 385–398.

8. S. Pandit et al., “NetProbe: A Fast and Scalable System for Fraud Detection in Online Auction Networks,” Proc. 16th Int’l Conf. World Wide Web (WWW 07), ACM Press, 2007, pp. 201–210.

9. B. Gavish and C.L. Tucci, “Reducing Internet Auction Fraud,” Comm. ACM, vol. 51, no. 5, 2008, pp. 89–97.

10. T. Kaszuba, A. Hupa, and A. Wierzbicki, “Comment Classification for Inter-net Auction Platforms,” Local Proc. 13th East-European Conf. (ADBIS 09), JUMI Publishing House, 2009, pp. 374–384.

References

Related documents

Before Use T1 T2 T3 S1 S2 S4 S3 T4 Drain hose Water inlet valve Power cord Back cover screws Back cover e n Control panel Start/Pause Door handle Work top Washer door Filter cover...

Following the Review and Approval Process for Ethically Responsible Research, the research proposal must be reviewed and approved by the Community Advisory Board, the

on behalf of the center for International Blood and Marrow Transplant Research, the Acute Leukemia Working Party and Eurocord (the European Group for Blood and Marrow

¾ Note: If you install PCI parallel ports to your system, you will be requested to restart your computer when you finish setting up each parallel port.. For example, if you

In short, –dramatic as it may sound– that logical form, which for the early Wittgenstein is constitutive of language, thought, and reality, is shown not just in propositions of

The meeting management system is expected to have a database for storing minutes of meetings and other information pertaining to meetings; Capture documents

The proposed method uti- lizes both local and global information, by expanding the receptive-field in the lowest level of the network, using dilated convolutions.. Five-fold

The reduction of the transmission heat loss is strongly dependent on the effective heat resistance of the insulating shutter, especially its ability to reduce the entrainment of