Amazon Mechanical Turk (AMT)
Amazon Mechanical Turk Workers Can Provide Consistent and Economically Meaningful Data
... use Amazon Mechanical Turk with ...on Amazon Mechanical Turk tends to be more male during the weekdays and the studies used to generate the data set that is analyzed here support ...
39
Gender differences in altruism on mechanical turk: Expectations and actual behaviour
... among Amazon Mechanical Turk (AMT) workers living in the ...US. AMT is an interesting platform to study gender differences in altruistic behaviour because AMT work- ers, although ...
15
Towards Extracting Medical Family History from Natural Language Interactions: A New Dataset and Baselines
... Our dataset consists of answers to a family history questionnaire collected from volunteers through Amazon Mechanical Turk (AMT) and from test cases of genetic counseling sessions (GCS). 2 ...
6
The average laboratory samples a population of 7,300 Amazon Mechanical Turk workers
... Amazon Mechanical Turk (MTurk) offers a large on-line workforce who complete human intelligence tasks (HITs). As experimenters, we can recruit these MTurk workers to complete our experiments and ...
16
Crowdsourcing samples in cognitive science
... marketplace, Amazon Mechanical Turk (MTurk), became a popular source of convenience samples for cognitive science research (MTurk terms are listed in Box ...
14
Semi supervised Word Alignment with Mechanical Turk
... from Amazon Mechanical Turk (MTurk) to help improve word alignment qual- ...by Turk- ers to that by experts, and incorporate the alignments in a semi-supervised word alignment tool to improve ...
5
Using the Amazon Mechanical Turk to Transcribe and Annotate Meeting Speech for Extractive Summarization
... Using Mechanical Turk for Annotations 5.1 Challenges of Using Mechanical Turk Unlike some other tasks that require little or no context in order to perform the annotation, summa- rization ...
9
Last Words: Amazon Mechanical Turk: Gold Mine or Coal Mine?
... “Mechanical Turk“ on 5 Novem- ber ...on Amazon Mechanical Turk (35 papers)—the existence of which is, in itself, strong evidence of the importance of the use of MTurk in the ...
8
Crowdsourced Comprehension: Predicting Prerequisite Structure in Wikipedia
... Accuracies of MaxEnt classifiers trained using the “leave one domain out” strategy are shown in Fig- ure 3; we report the test accuracy on each target do- main, as well as the average across domains. Perfor- mance of a ...
9
Active Learning with Efficient Feature Weighting Methods for Improving Data Quality and Classification Accuracy
... We conduct experiments on a Twitter dataset that contains tweets about TV shows and movies. The goal is to extract consumers’ emotional reactions to multimedia content, which has broad commer- cial applications including ...
9
User Modeling in Language Learning with Macaronic Texts
... We used Amazon Mechanical Turk (MTurk) to collect data. Users qualified for tasks by complet- ing a short quiz and survey about their language knowledge. Only users whose results indicated no ...
11
Amazon Mechanical Turk for Subjectivity Word Sense Disambiguation
... investigating Amazon Mechanical Turk (MTurk) as a source of non-expert natural language annotation, which is a cheap and quick alternative to expert annotations (Kaisser and Lowe, 2008; Mrozinski et ...
9
Shedding (a Thousand Points of) Light on Biased Language
... used Amazon Mechanical Turk judg- ments about sentences from American political blogs, asking annotators to indicate whether a sentence showed bias, and if so, in which political direction and ...
7
Separating Fact from Fear: Tracking Flu Infections on Twitter
... We used two Twitter data sets: a collection of 2 billion tweets from May 2009 and October 2010 (O’Connor et al., 2010) 4 and 1.8 billion tweets col- lected from August 2011 to November 2012. To obtain labeled data, we ...
7
Effects of Semantic Relatedness between Setups and Punchlines in Twitter Hashtag Games
... Despite their similarities, hashtag games offer several advantages over cartoon caption contests. First, setups are denoted using text-based hashtags, meaning they can be processed in a similar way to the responses. By ...
10
Error Driven Paraphrase Annotation using Mechanical Turk
... an Amazon Mechanical Turk study that explores this idea, and establishes via an oracle evaluation that it holds the potential to substantially improve translation ...
5
Dependency Tree Annotation with Mechanical Turk
... fully match the reference (FTM: full tree match rate). In the case of the Penn Treebank sentences, the reference is the automatic dependency conver- sion; for the Wikipedia sentences, we use the pri- mary author’s ...
5
Creating Speech and Language Data With Amazon’s Mechanical Turk
... In natural language processing, data drives re- search. The introduction of new large and widely accessible data sets creates whole new areas of re- search. There are many examples of such impact, the most famous of ...
12
The Mechanical Turk: Enduring Misapprehensions Concerning Artificial Intelligence
... in the minds of persons in general regarding the subject’ of Babbage’s work. ‘There is a misty notion amongst most of those who have attended at all to it,’ she writes, ‘that two “calculating machines” have been ...
26
Collecting Image Annotations Using Amazon’s Mechanical Turk
... matically correct but useless fragments such as “very dark” and “peace” only appear without a test. After requiring the qualification test, the major reasons for rejection by Turkers are typos such as in “The two dogs ...
9