we are dynamo: overcoming stalling and friction in collect - PowerPoint Presentation

Slide1

we

are dynamo: overcoming stalling and friction in collective action for crowd workers

@

niloufar_sniloufar@cs.stanford.eduSTANFORD HCI GROUP

Niloufar Salehi, Lilly

Irani

,

Michael

Bernstein,

Ali

Alkhatib

, Eva

Ogbe

, Kristy

Milland

,

ClickhappierSlide2

Many collective action efforts, online, never succeed.

[Hill 2011, Keegan et al. 2010, Kraut et al. 2012]Slide3
Slide4

Success cases:

Exception, rather than the norm. Slide5

decentralized characteristics

easy to gathermake it easy to disperseSlide6

The majority of collective action efforts fail.

failures successes [Hill 2011, Keegan et al. 2010, Kraut et al. 2012]Slide7

We engaged with crowd-workers on

Amazon Mechanical Turk.Slide8

dynamo

www.wearedynamo.orgSlide9

dynamo

Guidelines for Academic Requestershttp://guidelines.wearedynamo.org159 workers and 55 requesters signed.Slide10

dynamo

Letter Writing Campaignhttp://wearedynamo.org/dearjeffbezosSlide11

movements faced failure

Stalling: losing momentum

Friction: critiques and disagreements Slide12

challenges

dynamostalling & frictionlabor of actionSlide13

challenges

dynamostalling & frictionlabor of actionSlide14

challenges of online work

crowdsourcing platforms as computational infrastructure invisible workersstruggle to find good work or assert their rights[

Irani and Silberman 2013, Kittur et al. 2013]Slide15

turker collectives

resources for good work and mutual supportSlide16

turker collectives

resources for good work and mutual supportSlide17

forums struggled

open ended, sometimes unclear, goalsrequire public debatesharing of laborSlide18

what about a turker union?

Turkers found it impractical in their environment:high turnovershort work contractspersonal independence Slide19

How can we support an online collective not just to discuss, but to agree on a common goal and execute it?Slide20

challenges

dynamostalling & frictionlabor of actionSlide21

dynamo

www.wearedynamo.orgSlide22

interactive and iterativeengaged with Turkers

designed and built Dynamo togetherdynamo: our design processSlide23

interactive and iterativeengaged with Turkers

designed and built Dynamo togetherdynamo: our design processSlide24

532 registered Turkers22 ideas for action, 2 campaigns

7,000 unique visitors32,000 viewslongest discussion took 181 posts and had more than 1,800 viewsdynamo: activitySlide25

registration coderequires 100 tasks

prove worker statusone voicedynamo: registrationSlide26

anonymitynew space for

agreementrandomly generatedscreen names:e.g. light_dragonfly,excited_iguana, anddark_bird_of_paradisedynamo: registrationSlide27

1. assemble a collective

2. mobilizeSlide28

Guidelines for ethical research

dynamo: in actionSlide29

[

Irani and Silberman 2013]Slide30

ethical guidelines

Turkopticon abused by researcherresearchers lack exposure to Turkers’ issues and vulnerabilities Slide31

ethical guidelines

publicly available ethical guidelines:guide ethical behaviorback Turker claimsSlide32

ethical guidelines

from wanting ethical guidelines to creating itSlide33

ideaSlide34

✔

vote

28Slide35

mobilize

25 upvotes & more upvotes than downvotesdiscuss, take action, track progressSlide36

ethical guidelines

http://guidelines.wearedynamo.orgpublished on September 7th (Labor day)Slide37

ethical guidelines

http://guidelines.wearedynamo.org23 page write-upaccessed 43’000 timesfair payment, Turker privacy, etc.Slide38

ethical guidelines

support from researchers159 Turkers and 55 academic requestersinfo@wearedynamo.orgSlide39

ethical guidelinesSlide40

letter writing campaign

30 public letters at:http://wearedynamo.org/dearjeffbezosSlide41

letter writing campaign

30 public letters at:http://wearedynamo.org/dearjeffbezosSlide42

While both these campaigns were successful, they both almost failed.Slide43

challenges

dynamostalling & frictionlabor of actionSlide44

stalling

loosing momentumSlide45

stalling

example: letter writing campaign stalled“So, it seems no one is interested […]. [A Turker] just says we're doing it wrong, but won't say how to do it right, and no one else has input.” – A TurkerSlide46

stalling

example: letter writing campaign stalled“So, it seems no one is interested […]. [A Turker] just says we're doing it wrong, but won't say how to do it right, and no one else has input.” – A TurkerSlide47

friction

active criticism with negative emotionSlide48

friction

example: disappointment over one paragraph of the guidelines“I’m sorry to see the core document get excessively mired in technical detail in what purports to be a high-level document on ethics” – A TurkerSlide49

friction

Reflect arguments and suggest ways of addressing them.Turn friction into constructive criticism instead of flames.Slide50

twin pitfalls

Stalling: losing momentum

Friction: critiques and disagreements Slide51

twin pitfalls

Stalling: losing momentum

Friction: critiques and disagreements Slide52

Online collectives have more reasons to fail at acting together than to succeed.Slide53

challenges

dynamostalling & frictionlabor of actionSlide54

debates with deadlinesSlide55

debates with deadlines

“We've spent so much time and energy on this, we need a last effort to reach consensus […] We need your help for that to happen. Do you think setting a deadline would help?” – [Dynamo team]“I do agree we should wrap things up soon, if possible without unnecessary sacrifices.” – A TurkerSlide56

debates with deadlines

Participants know when they have to gear towards building consensus.Slide57

act and undoSlide58

act and undo

“the purpose of Dynamo is/should be to add to the rights/recourse of Turkers, not to limit/replace them.” – A TurkerSlide59

act and undo

Take action, but leave space for objections and undo your action if necessary.Slide60

labor of action

Structured mechanisms to guide an online collective to success.Slide61

HCI envisions how design and technology can a

ffect

positive changes in the worldSlide62

the social in social software

software

peopleSlide63

the social in social software

software

peopleSlide64

the social in social software

software

peopleSlide65

challenges

dynamostalling & frictionlabor of actionSlide66

Thank you!

http://wearedynamo.orgSpecial thanks to Turkers who participated in, and helped shape this project. This work was supported by a National Science Foundation award IIS-1351131, an Open Society Foundation grant, and a Stanford Graduate Fellowship.we are dynamo

@

niloufar_sniloufar@cs.stanford.eduSTANFORD HCI GROUP

Niloufar Salehi, Lilly

Irani

,

Michael

Bernstein,

Ali

Alkhatib

, Eva

Ogbe

, Kristy

Milland

,

Clickhappier