com Matthai Philipose Microsoft Research One Microsoft Way Redmond WA 98052 USA matthaipmicrosoftcom Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are n ID: 44137
Download Pdf The PPT/PDF document "Courteous Glass Jaeyeon Jung Microsoft R..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
facingintheirdailylives.Forexample,Autographer[1]isalightweightcamerathatcanbewornaroundtheneck.Ittakespicturesautomatically,capturingspontaneousimagesthatuserscanviewlater.Integratedwithpowerfulcomputingandnetworkingcapability,GoogleGlass[2]providesmuchmorethanhands-freevideorecordingsuchasaugmentedrealityfeaturesandvoicecommands.Therehavebeenalsovariousresearchprototypesthatattempttodevelopend-to-endapplicationsusingwearablecameras(e.g.,adietmonitoringsystem[5]).Privacyissuesassociatedwithtraditionalrecordingtechnologies(e.g.,CCTV,phoneswithcameras)stillremain,asdemonstratedbytheeaseofndingspycamwithasimpleonlinesearch.Asalways-onandtravelingwhereverawearergoes,theseissueswouldgetonlyworsewithwearablecameras.However,preventingunauthorizedrecordingisdicultevenwithlegalenforcementandtherearefewtechnicalsolutionsavailable(exceptaresearchprototypebyTruongetal.[7]).Inthispaper,wefocusonwell-intendedusersofwearablecamerasandtheprivacyissuesthattheyfacewhentheyencounterpeopleintheeldofview(FoV).Wearguethattheseprivacyissuesarisebecausewearablecamerasviolatesocialnormsthatpeopledevelopedaroundtheuseofhand-heldcameras.First,designedtobesmallandalmostcovert,wearablecamerasarehardtobenoticedbypeoplewhoareinFoV,thusdeprivingthemfromanopportunitytooptoutfromrecording(bywalkingoutsideFoVorcoveringtheface).Second,asthesecamerasareoftenlefton,eventhewearermaynotbeawareofrecordingandfailtoasktheconsenttothepeoplebeingrecorded.Indeed,Denningetal.reportthat17outof31participantstheyinterviewedexpressedthepreferenceforsomeonetoasktheirpermissionbeforerecordingthemwithaugmentedrealitydevices[4].Third,evenifthewearerrecognizesthatsomepeopleinFoVexpressedtheirpreferencenottoberecorded,itmaytakesomedelayandmaneuvertorespond(e.g.,oneneedstoholdabuttonfor6secondstoswitcho Autographer).Weproposethatitmaybeviabletomitigatetheseprivacyissueswithoutreducingthebenetsofcontinuousrecordingoftheeventsofinterestifwecanuselow-delitysensors,especiallyfar-infrared(FIR)imagerstomonitorforknownsocialcuessoastodeterminewhenisokaytoturnon/o RGBcameraswhichareusedtorunvisiontasks.Wearguethathavingaseparatesensordedicatedforenforcingprivacyruleswouldreducetheattacksurfaceandascapturingonlylow-delitythermalimages(ratherthanfull edgedRGBimages),FIRimagerscandosowiththeleastprivilege.Wepresentaninitialprototypeandadirectionforongoingresearchtofullydevelopawearablecamerathatiscourteous.SocialNormsWithRespectToRecordingIntheUS,di erentsetoflawsgovernaudioandvideorecordingsrespectively.Ourlimitedreviewoftheselaws(e.g.,http://www.rcfp.org/reporters-recording-guide)suggestthatrecordingaprivateconversationrequiresconsentfromoneorallpartiesbuttherequirementvariesstatebystate.However,videosurveillanceismostlyallowedwithoutconsentinpublicplaces(e.g.,shoppingmalls,citystreets)althoughsomestatesbantheuseofvideoorstillcameraswherethesubjecthasanexpectationofprivacy(e.g.,bathrooms).Althoughthelegallandscapeisamurkyandchanging,thereareafewsocialnormsthatpeoplehavebeenusinginordertoberespectfulforothers'privacypreferenceswhenshootingvideo: Opt-in:thecameraholderobtainsconsentfromthesubjectusuallythroughaverbalengagementbeforetakingapictureNotication:thecameraexplicitlysignalsrecording(withredlightorclickingsound)O -the-record(OTR):thesubjectsendssignalsofthedesirenottoberecordedeithertryingtoblockthecamera'sFoVorgettingoutoftheview.Aswediscussedintheprevioussection,wearablecamerastypicallyviolateallthesethreenormsastheyarecontinuouslycapturingscenestorunvarioustaskswhetherpeopleareinthesceneornot.Asaconsequence,wearablecamerasmayinadvertentlyrecordanddisclosepeoplesimagesagainsttheirwish,makingthemaccountablefortheiractionsandlocations(e.g.,iftheimagecapturedapersonatacompanyforacondentialjobinterview).NotethatourproposaliscomplementarytothesolutionproposedbyTemplemanetal.toblockcapturingblacklistedplaces[6].Tomitigatesuchprivacyissues,weproposethefollowingcourtesyprotocolforwearablecameras(Figure1).Webelievethattheprotocolwouldhaveaminimalimpactonmanytasksthatareproposedtorunonthesecamerasifthetasksdonotinvolvepeople(e.g.,remindingpeopletotakeapillbeforeeating,trackingthelastlocationwhereacarkeyisleft,keepingarecordofmealstaken).Fortheprotocoltobee ective,weassumeaprivacy-preservingoraclethattracksthepresenceofpeopleanddetectso -the-recordgestureswithoutrecordingvideoimages.1:while(true)f2:if(recordingison)f3:if(anewpersonentersintoFoV)4:turn off recording();5:elseif(OTRgestureisdetected)6:turn off recording();7:gelsef8:if(nopeoplearepresentinFoV)9:turnonrecording();10:elseif(anewpersonentersintoFoV)11:launchtheopt�inprocess();12:g13:gFigure1:Apseudocodeofthecourtesyprotocol:ThegoalistoturnonrecordingdevicesonlywhennopeoplearepresentinFoVorpeopleinFoVhaveconsentedforrecording.Nowweturntothequestionofhowtoimplementthisprivacy-preservingoracle.Firstwhentherecordingison(lines2to6),wemaypiggybackthetasksofdetectinganewperson(line3)ando -the-recordgestures(line5)usingstandardcomputervisionalgorithms.However,whentherecordingiso (lines8to11),weneedwaystodetectpeopleenteringinFoVwithoutrelyingonRGBcameras.Althoughinprinciple,onecanre-purposeRGBcameraswithsoftwarestacksthatlimittherecordingfunctionality,wearguethatsoftware-basedsolutionsarepronetoattacks(e.g.,[3]).Tominimizetheattacksurface,weproposethataseparatesensorshouldbeusedforenforcingtheprotocolwithhardware-levelisolation.Next,weoutlineoursolution.CourteousGlassFigure2showsourinitialhardwaremockup.Thewearablecameraisintegratedwithafar-infrared(FIR)imagerthatactsasaprivacy-preservingoraclediscussedintheprevioussection.Althoughcurrentlyhardware-basedcameraisolationistentativelyrepresentedwithawebcamcover,weenvisionthatalightweightcontrolcanbeeasily Figure4:RGBimages(top)vs.FIRimages(botton):Theimageontherightshowsthatao -the-recordgesture(tryingtoblockthecamera)canbealsodetectedusinganFIRimage.WenowexamineinmoredetailtheperformanceofFIRsensorsasa\gating"mechanismfordetectingpeople.Ideally,wewantgoodrecallandprecision:wewouldliketomaximizethefractionoftruefacesdetectedwhileminimizingthefractionofpixelsfalselydetectedasfaces.Theperformanceofthisschemewillofcoursedependonthepreciseface-detectionalgorithmappliedtotherawsensordata.Hereweconsiderasimpleschemewhereifthetemperatureofapixelexceedsaxedthreshold,weinferthatafaceispresentatthatpixelelseweinfernoface.Figure5showstheresultsfromasimpleexperimentestimatingthefractionofpixelsfromFIRfootagecollectedfrommanythousandframesfromwarmoutdoor(24Cambienttemperature),coldoutdoor(11C),coldindoorgarage(14C),indooroce(21C)andindoorlobby(19Cwithpeopleconstantlycominginfromcoldoutside)settings.Wepickasingletemperaturethresholdinthesesettingsandcomparethefractionoffacesforwhichatleastonepixelisabovethatthreshold(x-axis)tothefractionofpixelsthatarefalselydesignatedasfaces(y-axis).Atathresholdof85F(29.4C),forinstance,wedetect89%offaceswhileonlyallowing3%ofnon-facepixels.Thesenumbersrepresentearlyresultsusingaparticularlysimpledetector.Forinstance,thesepixelscouldgothroughanotherstageofface-detectionbasedontemperaturepatternsfortheface(asopposedtocrudesingle-thresholdrejection),yieldingfalsepositivefacesonceeveryhourorso.Further,the11%oftimeswhenwefailedtorecognizefaces,itwasoftenbecausethefaceswerefacingawayfromwearers.Whenfacesdirectlyfacethewearers,weexpectmuchhigherdetectionrates,missingperhaps0.1%ofallface-to-faceinteractions(basedonanalyzingourcurrentface-ondata). Figure5:Facedetectionratesvs.settingsInsummary,webelievethatFIRsensorsaresuitablefor