Realtime CrowdPowered Interfaces A Summary By Lance West S olve complex problems Crowd Latency gt 1 Minute Fastest workers start task 10 seconds after submission This is not real time This is nearly ID: 935592
Download Presentation The PPT/PDF document "Crowds in Two Seconds: Enabling" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Crowds in Two Seconds: Enabling Realtime Crowd-Powered Interfaces
A Summary By: Lance West
Slide2Solve complex problems
Crowd Latency >= 1 Minute
Fastest workers start task 10 seconds after submissionThis is not real time. This is “nearly realtime”
Crowd Labor Interfaces Today
Slide3Create “On-demand, synchronous crowds” “Retainer Model”
“Rapid Refinement”
Modifying Crowd Sourcing Interfaces
Slide4Hires crowd workers in advance and pays them a low stand-by rate until a task is ready. With this method, a majority of workers will begin work within 2 seconds. Over 75% in 3 seconds.
Retainer Model
Slide5Puppeteer: First responses in 2.1 secondsA|B – First 5 responses in 5-7 seconds
Puppeteer and A|B
Slide6As the first responders begin to show a consensus, the search space is narrowed for later responders. This decreases latency, increases reliability, and allows the possibility of having an initial guess in the first couple seconds of task submission, followed by a verified answer a few seconds later.
Rapid Refinement
Slide7Adrenaline
“When
at least 33% of the
workers have been in the same 25% of the timeline for at
least 2 seconds, Adrenaline declares agreement. These values
can be adjusted to trade off delay for false positives.
With these values, Adrenaline converges in 3–4 phases per
video. The first phase is the slowest, and agreement accelerates
as workers’ attention is focused on one area
.”
Slide8Works Cited
Bernstein, M. S., Brandt, J., Miller, R. C., &
Karger
, D. R. (
n.d.
). Retrieved November 13, 2014, from http://hci.stanford.edu/publications/2011/realtime/adrenaline-uist2011.pdf