amazon mechanical turk system
Clustering dictionary definitions using Amazon Mechanical Turk
... The identification of a list of senses for a given word in a corpus of documents is called word sense induction (WSI). SemEval 2007 and 2010 (SigLex, 2008) both evaluate WSI systems. The I2R system achieved the ...
9
Using Mechanical Turk to Annotate Lexicons for Less Commonly Used Languages
... In this work we present results from using Amazon’s Mechanical Turk (MTurk) to an- notate translation lexicons between English and a large set of less commonly used lan- guages. We generate candidate ...
6
Last Words: Amazon Mechanical Turk: Gold Mine or Coal Mine?
... on Amazon Mechanical Turk, in which non-expert evaluation of summarization systems is proved to be “not able to recover system rankings derived from ...
8
Effects of Semantic Relatedness between Setups and Punchlines in Twitter Hashtag Games
... Despite their similarities, hashtag games offer several advantages over cartoon caption contests. First, setups are denoted using text-based hashtags, meaning they can be processed in a similar way to the responses. By ...
10
User Modeling in Language Learning with Macaronic Texts
... used Amazon Mechanical Turk (MTurk) to collect ...SMT system was trained on the German-English Commoncrawl parallel text used in WMT 2015 (Bojar et ...
11
Semi supervised Word Alignment with Mechanical Turk
... using Amazon Mechanical Turk (MTurk) to collect bilin- gual word alignment data to assist automatic word align- ...a system including a word align- ment interface based on Javascript and a ...
5
Amazon Mechanical Turk for Subjectivity Word Sense Disambiguation
... Amazon Mechanical Turk (MTurk) is a mar- ketplace for so-called “human intelligence tasks” (HITs), or tasks that are easy for hu- mans but currently difficult for automated pro- ...SWSD system ...
9
Annotating Large Email Datasets for Named Entity Recognition with Mechanical Turk
... feedback system was designed to improve the transparency of the compensation system and motivate higher quality work over ...the system achieved the desired ...
9
Collecting Semantic Data from Mechanical Turk for a Lexical Knowledge Resource in a Text to Picture Generating System
... Data collected from AMT tasks was manually filtered via removal of undesirable target item-response item pairs and classified via definition of the relations between the remaining target item-response item pairs. ...
5
Collecting Image Annotations Using Amazon’s Mechanical Turk
... can predict not just the presence and location of cer- tain objects in an image, but also the relations be- tween objects, their attributes, or the actions and events they participate in. Such information can neither be ...
9
An Innovative Crowdsourcing Approach for Amazon Mechanical Turk
... for Amazon Mechanical Turk (AMT) crowdsourcing ...reputation system; some workers may cheat on tasks in order to maximize their income, as a result requesters are uncertain of the quality of ...
6
The Language Demographics of Amazon Mechanical Turk
... translation system that pro- duced higher quality translations of dialect Arabic than a system a trained on 100 times more Mod- ern Standard Arabic-English parallel ...MT system on crowdsourced ...
14
Error Driven Paraphrase Annotation using Mechanical Turk
... translation system is typically only one of many ways the input sentence could have been expressed, and alternative forms of expression can often produce a better trans- ...translation system. We report on ...
5
Exploring Normalization Techniques for Human Judgments of Machine Translation Adequacy Collected Using Amazon Mechanical Turk
... Human judgments of translation quality play a vital role in the development of effective machine trans- lation (MT) systems. Such judgments can be used to measure system quality in evaluations (Callison- Burch et ...
5
Active Learning with Amazon Mechanical Turk
... We therefore designed our own system for anno- tation experiments. It consists of a two-tiered ap- plication architecture. The frontend tier is a web application that serves two purposes. First, the ad- ...
11
Creating a Research Collection of Question Answer Sentence Pairs with Amazon’s Mechanical Turk
... Here, Question 1395 (Who is Tom Cruise married to?) has been answered with the string “Nicole Kidman”. NYT19990326.0303 is the identifier of one particular document in the AQUAINT corpus. The third column (“1”), in this ...
5
Creating Speech and Language Data With Amazon’s Mechanical Turk
... Mechanical Turk. It shows that most US-based workers use Me- chanical Turk for secondary income purposes (to have spending money for hobbies or going out), but that the overwhelming majority of them ...
12
2018_Beers.pdf
... used Amazon Mechanical Turk to recruit participants to take a Qualtrics ...survey. Amazon Mechanical Turk is a crowdsourcing web service that coordinates the activities of ...
12
Crowdsourcing Document Relevance Assessment with Mechanical Turk
... Crowd demographics. While it is typically sug- gested that experts produce superior annotations, there are important questions of effects from who is judging the annotations. For example, if you want to know if the ...
8
The Mechanical Turk: Enduring Misapprehensions Concerning Artificial Intelligence
... chines is quite so pronounced. As the sentient CPU at the heart of such a ma- chine, Turing would often have found himself performing operations, following his own written instructions, that he could never have ...
26