Gajos School of Engineering and Applied Sciences Harvard University Cambridge MA Abstract Curio is a platform that enables researchers in the sci ences and humanities who are domain experts but not necessarily technically savvy to create monitor and ID: 44942
Download Pdf The PPT/PDF document "Curio A Platform for Supporting MixedExp..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
thoseofotherplatforms,throughwhichparticipantscollectandannotatedatainanopen-endedway,notknowingex-actlyhowandwhenthedatawillbeused.Ourquestion-drivenapproachisinspiritsimilartoarecentlylaunchedZooniverseprojectcalled“GalaxyZooQuench”(http://quench.galaxyzoo.org),wheretheprojecthasalimitedtimescaleandisdrivenbyaresearchpurpose.DesigningTasksandWorkowsManyresearchersareend-users,asopposedtocreators,ofwebtechnology.Assuch,theprocessofdesigningatask–i.e.,reasoningaboutandcreatingtheowofactivitiesandthecorrespondinguserinterfacedesigns–canbechal-lenging.OnCurio,researcherscreatetasksthroughasetofguidedquestions,focusingtheirattentiononspecifyingwhattheywantparticipantstoaccomplish,asopposedtodesigninguserinterface.Certaintasksarecomplex,requiringdecompositionintosimplersubtasksandchainingofthesesubtasksintoawork-ow.Forexample,inordertoclassifymedicalimagesortimeseriesdata,expertsuseasetofcomplexrulesthattakeyearstointernalize.Inthesecases,insteadofrequiringtheexperttoexplicitlydecomposetheclassicationtaskortospecifyanalgorithm,Curioasksexpertstoexpresstheirknowledgebydemonstratingtheirreasoningonconcreteex-amples,andautomaticallyinducestheunderlyingworkow.MonitoringandReningCuriosupportsaniterativeapproachtocrowdsourcing,whererequesters(i.e.,researchers)monitorthesolutionasitisbeingassembledandintervenetoimprovethecompu-tationaloutcomes,e.g.,byclarifyingthetaskinstructions,providingadditionalexamples,orcommunicatingwiththeworkers.Iterationsareneededespeciallybecauseexpertsof-tenover-estimateorunder-estimatethecapabilitiesofnon-experts,andmayunknowinglyusejargoninthetaskinstruc-tionsthatisinaccessibletonovices.Inaddition,researchisanon-linearprocess:theinterestsandneedsoftheresearchercanchangeovertime.Certaintasksmaybecomehigherorlowerontheprioritylist,astheresearcherreneshisques-tionandapproach.Curioprovidesseveralalgorithmsforin-ferringqualityintheabsenceofgroundtruth,datavisualiza-tionstosupportresearchersinidentifyingproblematicstagesintheworkowandtheexibilitytomodifyvariousaspectsoftheprojects,withoutmajordisruptionfromthepartici-pants'pointsofview.CommunicatingwithParticipantsInpreviousstudiesofcitizenscienceplatforms(Rotmanetal.2012),itwasfoundthatthereisadisconnectbe-tweencitizenscientistsandscientists.Citizenscientistsmaybeinitiallymotivatedtohelp,buttheyoftenndscientists“unfriendly”andlosemotivationwhentheylosetrackofwhytheirworkmatters.OntheCurioplatform,wefacili-tateresearcher-participantcommunicationinmultipleways.First,asparticipantsarecompletingtasks,ourplatformin-termittentlyshowsthemvisualizationsthatemphasizetheiruniquecontributionaswellaspartialresultstotheresearchquestion.Second,theplatformprovidesavotingsystemthatallowsresearchersandparticipantstodebateambigu-ouscasesinamoretargetedwaythanopen-endedforumdiscussions.Finally,researchersareencouragedtospecifyprizesthatwillappealtothespecicinterestsofthecontrib-utors,andparticipantsearnprizesbycompletingtasksinanaccurateandefcientway.SharingDataOnechallengeinquestion-drivencitizenscienceisthatboththeresearchintentanddataispubliclyexposedbeforethepublicationstage.Dependingontheparticularresearchdis-ciplineanditsexistingdatasharingpractices,researchersmayormaynotbecomfortablemakingtheirworkexter-nallyvisibletooearlyintheresearchprocess.Toaddressthis,Curioprovidesanoptionforresearcherstokeeptheirprojectprivateandbyinvitationonly,andallowsresearcherstoschedulethereleaseoftheirdata,thusencouragingshar-ingofresourceswithoutcompromisingtheneedforprivacy.FormingHierarchicalTeamsBeyondsharingdata,researchersareoftenreluctanttoas-signmajorresponsibilitiestonovices,whomtheydonotyettrusttohaveenoughexpertiseandexperience.Toaddressthis,Curiomimicstheexistinghierarchicalcultureandqual-itycontrolpracticesinacademia,wheremoreseniormem-berssupervisethosewhoarelessexperienced,andmembersofaresearchteammeetinformallytodiscussanddecipherambiguousdata.OnCurio,researcherscanspecifyaprojecttobeaccessibletoparticipantsbyinvitationonly.Thises-sentiallyallowseachresearchertocreatehisowncrowdandselectparticipantsbasedonhisuniquesetofexpertiserequirementsandotherdifcult-to-articulatecriteria,e.g.,trust,accountabilityandpersonality.Researcherscanalsocaninvitecollaboratorswithsubstantialdomainknowledge(e.g.,graduatestudents,postdocs,colleaguesandadvancedcitizenscientists)toformamanagementteam,tocollabora-tivelymonitoranddebugthecrowdsourcingprocess.ConclusionWeexpecttheCurioplatformtolaunchinthefallandhostseveralprojects,e.g.,whereparticipantsannotatemuseumrecordstoevaluatetheeffectsofclimatechangeonower-ingtimes,anddesignneworganicmoleculesforpoweringsolarcells.Asahumancomputationsystem,Curioraisesexcitingnewresearchopportunitiesandchallenges.ReferencesRotman,D.;Preece,J.;Hammock,J.;Procita,K.;Hansen,D.;Parr,C.;Lewis,D.;Jacobs,D.;andBiswas,A.2012.Dynamicchangesinmotivationincollaborativecitizensci-enceprojects.InCSCW.AcknowledgmentsWeespeciallythankAnnaKiyantseva,ZachHamed,LaszloSeress,andthenumerousresearchersfortheirhelpdevelop-ingtheCurioidea.ThisresearchissupportedinpartbytheNationalScienceFoundationDEB-1208256.