Quality Management and other stories Panos Ipeirotis New York University amp Tagasauris Amazon Mechanical Turk patent pending Tagging patent pending But what if MTurk workers ID: 209818
Download Presentation The PPT/PDF document "Crowdsourcing" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Crowdsourcing
Quality
Management
and other stories
Panos
Ipeirotis
New York University &
TagasaurisSlide2
Amazon Mechanical TurkSlide3
patent pending
TaggingSlide4
patent pending
But what if
MTurk
workers
submit
incorrect
tags?Slide5
VotingSlide6
Voting
But what if
MTurk
workers
also vote
incorrectly
?Slide7
Voting
But what if
MTurk
workers
also vote
incorrectly
?
Redundancy
: 5+ votes per tagSlide8
But my
task
is
not just right or wrong…
Creation
HIT
(e.g
.
describe the image)
Improve HIT
(e.g. improve description)
Compare HIT (voting)
Which is better?
Describe thisSlide9
version 1:
A parial view of a pocket calculator together with some coins and a pen.
version 2: A view of personal items a calculator, and some gold and copper coins, and a round tip pen, these are all pocket
and wallet sized item used for business, writting, calculating prices or solving math problems and purchasing items.version 3:
A close-up photograph of the following items: A CASIO multi-function calculator. A ball point pen, uncapped. Various coins, apparently European, both copper and gold. Seems to be a theme illustration for a brochure or document cover treating finance, probably personal finance.version 4:
…Various British coins; two of £1 value, three of 20p value and one of 1p value. …version 8:
“A close-up photograph of the following items: A CASIO multi-function, solar powered scientific calculator. A blue ball point pen with a blue rubber grip and the tip extended. Six British coins; two of £1 value, three of 20p value and one of 1p value. Seems to be a theme illustration for a brochure or document cover treating finance - probably personal finance."Slide10
New York: Old
and Newhttp://newyorkoldandnew.blogspot.com/
An engagement experiment:
“Search the NYPL digital library for an old photo of New York, and then find a photo of the same location, today
”Slide11
1935Slide12
2007Slide13
1933Slide14
2007Slide15
1890Slide16
2007Slide17
1890Slide18
1900Slide19
1976Slide20
2008Slide21
The incomplete knowledge graph
[
Harsen Homestead, 72nd Street, ca. 1888.]Slide22
The incomplete knowledge graph
Harsen
Homestead
???Slide23
The incomplete knowledge graph
Jacob
Harsen
, a farmer who
started
Harsenville, a village spanning 68th Street to 81st Street between Central Park West and the Hudson River.
Source: http
://
www.oneshermansquare.com/HistoryofShermanSquare.aspx Slide24
The incomplete knowledge graph
Crowd not only annotates historical archives but also
contributes back
to the knowledge graph
[allowing easier annotation in the future]Slide25
New York
Times, 1911Slide26
The incomplete knowledge graph
Crowd participation not only
annotates historical archives
but also
contributes back
to the knowledge graph [allowing easier annotation in the future]