Figure1:CryptDB'sarchitectureconsistingoftwoparts:adatabaseproxyandanu - PDF document

Figure1:CryptDB'sarchitectureconsistingoftwoparts:adatabaseproxyandanunmodiedDBMS.CryptDBusesuser-denedfunctions(UDFs)toperformcryptographicoperationsintheDBMS.Rectangularandroundedboxesrepresentprocessesanddata,respectively.ShadingindicatescomponentsaddedbyCryptDB.Dashedlinesindicateseparationbetweenusers'computers,theapplicationserver,aserverrunningCryptDB'sdatabaseproxy(whichisusuallythesameastheapplicationserver),andtheDBMSserver.CryptDBaddressestwokindsofthreats,shownasdottedlines.Inthreat1,acuriousdatabaseadministratorwithcompleteaccesstotheDBMSserversnoopsonprivatedata,inwhichcaseCryptDBpreventstheDBAfromaccessinganyprivateinformation.Inthreat2,anadversarygainscompletecontroloverboththesoftwareandhardwareoftheapplication,proxy,andDBMSservers,inwhichcaseCryptDBensurestheadversarycannotobtaindatabelongingtousersthatarenotloggedin(e.g.,user2).well-denedsetofprimitiveoperators,suchasequalitychecks,ordercomparisons,aggregates(sums),andjoins.Byadapt-ingknownencryptionschemes(forequality,additions,andor-derchecks)andusinganewprivacy-preservingcryptographicmethodforjoins,CryptDBencryptseachdataiteminawaythatallowstheDBMStoexecuteonthetransformeddata.CryptDBisefcientbecauseitmostlyusessymmetric-keyencryption,avoidsfullyhomomorphicencryption,andrunsonunmodiedDBMSsoftware(byusinguser-denedfunctions).Thesecondtechniqueisadjustablequery-basedencryption.SomeencryptionschemesleakmoreinformationthanothersaboutthedatatotheDBMSserver,butarerequiredtoprocesscertainqueries.ToavoidrevealingallpossibleencryptionsofdatatotheDBMSapriori,CryptDBcarefullyadjuststheSQL-awareencryptionschemeforanygivendataitem,dependingonthequeriesobservedatrun-time.Toimplementtheseadjust-mentsefciently,CryptDBusesonionsofencryption.Onionsareanovelwaytocompactlystoremultipleciphertextswithineachotherinthedatabaseandavoidexpensivere-encryptions.Thethirdideaistochainencryptionkeystouserpasswords,sothateachdataiteminthedatabasecanbedecryptedonlythroughachainofkeysrootedinthepasswordofoneoftheuserswithaccesstothatdata.Asaresult,iftheuserisnotloggedintotheapplication,andiftheadversarydoesnotknowtheuser'spassword,theadversarycannotdecrypttheuser'sdata,eveniftheDBMSandtheapplicationserverarefullycompromised.Toconstructachainofkeysthatcapturestheapplication'sdataprivacyandsharingpolicy,CryptDBallowsthedevelopertoprovidepolicyannotationsovertheapplication'sSQLschema,specifyingwhichusers(orotherprincipals,suchasgroups)haveaccesstoeachdataitem.WehaveimplementedCryptDBonbothMySQLandPostgres;ourdesignandmostofourimplementationshouldbeapplicabletomoststandardSQLDBMSes.Ananalysisofa10-daytraceof126millionSQLqueriesfrommanyapplicationsatMITsuggeststhatCryptDBcansupportoperationsoverencrypteddatafor99.5%ofthe128,840columnsseeninthetrace.OurevaluationshowsthatCryptDBhaslowoverhead,reducingthroughputby14.5%forthephpBBwebforumapplication,andby26%forqueriesfromTPC-C,comparedtounmodiedMySQL.WeevaluatedthesecurityofCryptDBonsixrealapplications(includingphpBB,theHotCRPconferencemanagementsoftware[27],andtheOpenEMRmedicalrecordsapplication);theresultsshowthatCryptDBprotectsmostsensitiveeldswithhighlysecureencryptionschemes.Chainingencryptionkeystouserpasswordsrequires11–13uniqueschemaannotationstoenforceprivacypoliciesonmorethan20sensitiveelds(includinganewpolicyinHotCRPforhandlingpapersinconictwithaPCchair)and2–7linesofsourcecodechangesforthreemulti-userwebapplications.Therestofthispaperisstructuredasfollows.Inx2,wediscussthethreatsthatCryptDBdefendsagainstinmoredetail.Then,wedescribeCryptDB'sdesignforencryptedqueryprocessinginx3andforkeychainingtouserpasswordsinx4.Inx5,wepresentseveralcasestudiesofhowapplicationscanuseCryptDB,andinx6,wediscusslimitationsofourdesign,andwaysinwhichitcanbeextended.Next,wedescribeourprototypeimplementationinx7,andevaluatetheperformanceandsecurityofCryptDB,aswellastheeffortrequiredforapplicationdeveloperstouseCryptDB,inx8.WecompareCryptDBtorelatedworkinx9andconcludeinx10.2SECURITYOVERVIEWFigure1showsCryptDB'sarchitectureandthreatmodels.CryptDBworksbyinterceptingallSQLqueriesinadatabaseproxy,whichrewritesqueriestoexecuteonencrypteddata(CryptDBassumesthatallqueriesgothroughtheproxy).Theproxyencryptsanddecryptsalldata,andchangessomequeryoperators,whilepreservingthesemanticsofthequery.TheDBMSserverneverreceivesdecryptionkeystotheplaintextsoitneverseessensitivedata,ensuringthatacuriousDBAcannotgainaccesstoprivateinformation(threat1).Toguardagainstapplication,proxy,andDBMSservercompro-mises(threat2),developersannotatetheirSQLschematodenedifferentprincipals,whosekeyswillallowdecryptingdifferentpartsofthedatabase.Theyalsomakeasmallchangetotheirapplicationstoprovideencryptionkeystotheproxy,asdescribedinx4.Theproxydetermineswhatpartsofthedatabaseshouldbeencryptedunderwhatkey.TheresultisthatCryptDBguaranteesthecon-dentialityofdatabelongingtousersthatarenotloggedinduringacompromise(e.g.,user2inFigure1),andwhodonotloginuntilthecompromiseisdetectedandxedbytheadministrator.AlthoughCryptDBprotectsdatacondentiality,itdoesnotensuretheintegrity,freshness,orcompletenessofresultsreturnedtotheapplication.Anadversarythatcompromisestheapplication,proxy,orDBMSserver,oramaliciousDBA,candeleteanyorallofthedatastoredinthedatabase.Similarly,attacksonusermachines,suchascross-sitescripting,areoutsideofthescopeofCryptDB.WenowdescribethetwothreatmodelsaddressedbyCryptDB,andthesecurityguaranteesprovidedunderthosethreatmodels.2.1Threat1:DBMSServerCompromiseInthisthreat,CryptDBguardsagainstacuriousDBAorotherexter-nalattackerwithfullaccesstothedatastoredintheDBMSserver.Ourgoaliscondentiality(datasecrecy),notintegrityoravailability.Theattackerisassumedtobepassive:shewantstolearncondential2 fortheoriginalquery,exceptthattheoperatorscomprisingthequery,suchasselections,projections,joins,aggregates,andorderings,areperformedonciphertexts,andusemodiedoperatorsinsomecases.CryptDB'sproxystoresasecretmasterkeyMK,thedatabaseschema,andthecurrentencryptionlayersofallcolumns.TheDBMSserverseesananonymizedschema(inwhichtableandcol-umnnamesarereplacedbyopaqueidentiers),encrypteduserdata,andsomeauxiliarytablesusedbyCryptDB.CryptDBalsoequipstheserverwithCryptDB-specicuser-denedfunctions(UDFs)thatenabletheservertocomputeonciphertextsforcertainoperations.ProcessingaqueryinCryptDBinvolvesfoursteps:1.Theapplicationissuesaquery,whichtheproxyinterceptsandrewrites:itanonymizeseachtableandcolumnname,and,usingthemasterkeyMK,encryptseachconstantinthequerywithanencryptionschemebestsuitedforthedesiredoperation(x3.1).2.TheproxychecksiftheDBMSservershouldbegivenkeystoadjustencryptionlayersbeforeexecutingthequery,andifso,issuesanUPDATEqueryattheDBMSserverthatinvokesaUDFtoadjusttheencryptionlayeroftheappropriatecolumns(x3.2).3.TheproxyforwardstheencryptedquerytotheDBMSserver,whichexecutesitusingstandardSQL(occasionallyinvokingUDFsforaggregationorkeywordsearch).4.TheDBMSserverreturnsthe(encrypted)queryresult,whichtheproxydecryptsandreturnstotheapplication.3.1SQL-awareEncryptionWenowdescribetheencryptiontypesthatCryptDBuses,includinganumberofexistingcryptosystems,anoptimizationofarecentscheme,andanewcryptographicprimitiveforjoins.Foreachencryptiontype,weexplainthesecuritypropertythatCryptDBrequiresfromit,itsfunctionality,andhowitisimplemented.Random(RND).RNDprovidesthemaximumsecurityinCryptDB:indistinguishabilityunderanadaptivechosen-plaintextattack(IND-CPA);theschemeisprobabilistic,meaningthattwoequalvaluesaremappedtodifferentciphertextswithoverwhelmingprobability.Ontheotherhand,RNDdoesnotallowanycompu-tationtobeperformedefcientlyontheciphertext.AnefcientconstructionofRNDistouseablockcipherlikeAESorBlowshinCBCmodetogetherwitharandominitializationvector(IV).(WemostlyuseAES,exceptforintegervalues,whereweuseBlowshforits64-bitblocksizebecausethe128-bitblocksizeofAESwouldcausetheciphertexttobesignicantlylonger).Since,inthisthreatmodel,CryptDBassumestheserverdoesnotchangeresults,CryptDBdoesnotrequireastrongerIND-CCA2construction(whichwouldbesecureunderachosen-ciphertextattack).However,itwouldbestraightforwardtouseanIND-CCA2-secureimplementationofRNDinstead,suchasablockcipherinUFEmode[13],ifneeded.Deterministic(DET).DEThasaslightlyweakerguarantee,yetitstillprovidesstrongsecurity:itleaksonlywhichencryptedvaluescorrespondtothesamedatavalue,bydeterministicallygeneratingthesameciphertextforthesameplaintext.Thisencryptionlayerallowstheservertoperformequalitychecks,whichmeansitcanperformselectswithequalitypredicates,equalityjoins,GROUPBY,COUNT,DISTINCT,etc.Incryptographicterms,DETshouldbeapseudo-randompermu-tation(PRP)[20].For64-bitand128-bitvalues,weuseablockcipherwithamatchingblocksize(BlowshandAESrespectively);wemaketheusualassumptionthattheAESandBlowshblockciphersarePRPs.Wepadsmallervaluesoutto64bits,butfordatathatislongerthanasingle128-bitAESblock,thestandardCBCmodeofoperationleaksprexequality(e.g.,iftwodataitemshaveanidenticalprexthatisatleast128bitslong).Toavoidthisproblem,weuseAESwithavariantoftheCMCmode[24],whichcanbeapproximatelythoughtofasoneroundofCBC,followedbyanotherroundofCBCwiththeblocksinthereverseorder.SincethegoalofDETistorevealequality,weuseazeroIV(or“tweak”[24])forourAES-CMCimplementationofDET.Order-preservingencryption(OPE).OPEallowsorderrela-tionsbetweendataitemstobeestablishedbasedontheiren-cryptedvalues,withoutrevealingthedataitself.Ifxy,thenOPEK(x)OPEK(y),foranysecretkeyK.Therefore,ifacolumnisencryptedwithOPE,theservercanperformrangequerieswhengivenencryptedconstantsOPEK(c1)andOPEK(c2)correspondingtotherange[c1;c2].TheservercanalsoperformORDERBY,MIN,MAX,SORT,etc.OPEisaweakerencryptionschemethanDETbecauseitrevealsorder.Thus,theCryptDBproxywillonlyrevealOPE-encryptedcolumnstotheserverifusersrequestorderqueriesonthosecolumns.OPEhasprovablesecurityguarantees[4]:theencryptionisequiva-lenttoarandommappingthatpreservesorder.Theschemeweuse[4]istherstprovablysecuresuchscheme.UntilCryptDB,therewasnoimplementationnoranymeasureofthepracticalityofthescheme.Thedirectimplementationoftheschemetook25msperencryptionofa32-bitintegeronanIntel2.8GHzQ9550processor.WeimprovedthealgorithmbyusingAVLbinarysearchtreesforbatchencryption(e.g.,databaseloads),reducingthecostofOPEencryptionto7msperencryptionwithoutaffectingitssecurity.WealsoimplementedahypergeometricsamplerthatliesatthecoreofOPE,portingaFortranimplementationfrom1988[25].Homomorphicencryption(HOM).HOMisasecureprobabilis-ticencryptionscheme(IND-CPAsecure),allowingtheservertoperformcomputationsonencrypteddatawiththenalresultde-cryptedattheproxy.Whilefullyhomomorphicencryptionispro-hibitivelyslow[10],homomorphicencryptionforspecicoperationsisefcient.Tosupportsummation,weimplementedthePailliercryptosystem[35].WithPaillier,multiplyingtheencryptionsoftwovaluesresultsinanencryptionofthesumofthevalues,i.e.,HOMK(x)
HOMK(y)=HOMK(x+y),wherethemultiplicationisperformedmodulosomepublic-keyvalue.TocomputeSUMaggre-gates,theproxyreplacesSUMwithcallstoaUDFthatperformsPailliermultiplicationonacolumnencryptedwithHOM.HOMcanalsobeusedforcomputingaveragesbyhavingtheDBMSserverreturnthesumandthecountseparately,andforincrementingvalues(e.g.,SETid=id+1),onwhichweelaborateshortly.WithHOM,theciphertextis2048bits.Intheory,itshouldbepossibletopackmultiplevaluesfromasinglerowintooneHOMciphertextforthatrow,usingtheschemeofGeandZdonik[17],whichwouldresultinanamortizedspaceoverheadof2(e.g.,a32-bitvalueoccupies64bits)foratablewithmanyHOM-encryptedcolumns.However,wehavenotimplementedthisoptimizationinourprototype.Thisoptimizationwouldalsocomplicatepartial-rowUPDATEoperationsthatresetsome—butnotall—ofthevaluespackedintoaHOMciphertext.Join(JOINandOPE-JOIN).Aseparateencryptionschemeisnecessarytoallowequalityjoinsbetweentwocolumns,becauseweusedifferentkeysforDETtopreventcross-columncorrelations.JOINalsosupportsalloperationsallowedbyDET,andalsoen-ablestheservertodeterminerepeatingvaluesbetweentwocolumns.OPE-JOINenablesjoinsbyorderrelations.Weprovideanewcryp-tographicschemeforJOINandwediscussitinx3.4.4 EmployeesID Name 23 AliceTable1C1-IV C1-Eq C1-Ord C1-Add C2-IV C2-Eq C2-Ord C2-Search x27c3 x2b82 xcb94 xc2e4 x8a13 xd1e3 x7eb1 x29b0Figure3:Datalayoutattheserver.Whentheapplicationcreatesthetableshownontheleft,thetablecreatedattheDBMSserveristheoneshownontheright.Ciphertextsshownarenotfull-length.checkisrequestedonacolumnandtheserverbringsthecolumntolayerDET,thecolumnremainsinthatstate,andfuturequerieswithequalitychecksrequirenodecryption.ThispropertyistheinsightintowhyCryptDB'soverheadismodestinthesteadystate(seex8):theservermostlyperformstypicalSQLprocessing.3.3ExecutingoverEncryptedDataOncetheonionlayersintheDBMSareatthelayernecessarytoexecuteaquery,theproxytransformsthequerytooperateontheseonions.Inparticular,theproxyreplacescolumnnamesinaquerywithcorrespondingonionnames,basedontheclassofcomputationperformedonthatcolumn.Forexample,fortheschemashowninFigure3,areferencetotheNamecolumnforanequalitycomparisonwillbereplacedwithareferencetotheC2-Eqcolumn.Theproxyalsoreplaceseachconstantinthequerywithacorre-spondingonionencryptionofthatconstant,basedonthecompu-tationinwhichitisused.Forinstance,ifaquerycontainsWHEREName=`Alice',theproxyencrypts`Alice'bysuccessivelyap-plyingallencryptionlayerscorrespondingtoonionEqthathavenotyetbeenremovedfromC2-Eq.Finally,theserverreplacescertainoperatorswithUDF-basedcounterparts.Forinstance,theSUMaggregateoperatorandthe+column-additionoperatormustbereplacedwithaninvocationofaUDFthatperformsHOMadditionofciphertexts.Equalityandorderoperators(suchas=and)donotneedsuchreplacementandcanbeapplieddirectlytotheDETandOPEciphertexts.Oncetheproxyhastransformedthequery,itsendsthequerytotheDBMSserver,receivesqueryresults(consistingofencrypteddata),decryptstheresultsusingthecorrespondingonionkeys,andsendsthedecryptedresulttotheapplication.Readqueryexecution.Tounderstandqueryexecutionoverci-phertexts,considertheexampleschemashowninFigure3.Initially,eachcolumninthetableisdressedinallonionsofencryption,withRND,HOM,andSEARCHasoutermostlayers,asshowninFig-ure2.Atthispoint,theservercanlearnnothingaboutthedataotherthanthenumberofcolumns,rows,anddatasize.Toillustratewhenonionlayersareremoved,considerthequery:SELECTIDFROMEmployeesWHEREName=`Alice',whichrequiresloweringtheencryptionofNametolayerDET.Toexecutethisquery,theproxyrstissuesthequeryUPDATETable1SETC2-Eq=DECRYPT RND(KT1;C2;Eq;RND,C2-Eq,C2-IV),wherecolumnC2correspondstoName.TheproxythenissuesSELECTC1-Eq,C1-IVFROMTable1WHEREC2-Eq=x7..d,wherecolumnC1correspondstoID,andwherex7..distheEqonionencryptionof“Alice”withkeysKT1;C2;Eq;JOINandKT1;C2;Eq;DET(seeFigure2).NotethattheproxymustrequesttherandomIVfromcolumnC1-IVinordertodecrypttheRNDciphertextfromC1-Eq.Finally,theproxydecryptstheresultsfromtheserverusingkeysKT1;C1;Eq;RND,KT1;C1;Eq;DET,andKT1;C1;Eq;JOIN,obtainstheresult23,andreturnsittotheapplication.IfthenextqueryisSELECTCOUNT(*)FROMEmployeesWHEREName=`Bob',noserver-sidedecryptionsarenecessary,andtheproxydirectlyissuesthequerySELECTCOUNT(*)FROMTable1WHEREC2-Eq=xbb..4a,wherexbb..4aistheEqonionencryptionof“Bob”usingKT1;C2;Eq;JOINandKT1;C2;Eq;DET.Writequeryexecution.TosupportINSERT,DELETE,andUPDATEqueries,theproxyappliesthesameprocessingtothepredi-cates(i.e.,theWHEREclause)asforreadqueries.DELETEqueriesre-quirenoadditionalprocessing.ForallINSERTandUPDATEqueriesthatsetthevalueofacolumntoaconstant,theproxyencryptseachinsertedcolumn'svaluewitheachonionlayerthathasnotyetbeenstrippedoffinthatcolumn.TheremainingcaseisanUPDATEthatsetsacolumnvaluebasedonanexistingcolumnvalue,suchassalary=salary+1.SuchanupdatewouldhavetobeperformedusingHOM,tohandleaddi-tions.However,indoingso,thevaluesintheOPEandDETonionswouldbecomestale.Infact,anyhypotheticalencryptionschemethatsimultaneouslyallowsadditionanddirectcomparisonontheciphertextisinsecure:ifamaliciousservercancomputetheorderoftheitems,andcanincrementthevaluebyone,theservercanrepeatedlyaddonetoeacheldhomomorphicallyuntilitbecomesequaltosomeothervalueinthesamecolumn.Thiswouldallowtheservertocomputethedifferencebetweenanytwovaluesinthedatabase,whichisalmostequivalenttoknowingtheirvalues.Therearetwoapproachestoallowupdatesbasedonexistingcolumnvalues.Ifacolumnisincrementedandthenonlyprojected(nocomparisonsareperformedonit),thesolutionissimple:whenaqueryrequeststhevalueofthiseld,theproxyshouldrequesttheHOMciphertextfromtheAddonion,insteadofciphertextsfromotheronions,becausetheHOMvalueisup-to-date.Forinstance,thisapproachappliestoincrementqueriesinTPC-C.Ifacolumnisusedincomparisonsafteritisincremented,thesolutionistoreplacetheupdatequerywithtwoqueries:aSELECToftheoldvaluestobeupdated,whichtheproxyincrementsandencryptsaccordingly,followedbyanUPDATEsettingthenewvalues.Thisstrategywouldworkwellforupdatesthataffectasmallnumberofrows.OtherDBMSfeatures.MostotherDBMSmechanisms,suchastransactionsandindexing,workthesamewaywithCryptDBoverencrypteddataastheydooverplaintext,withnomodications.Fortransactions,theproxypassesalonganyBEGIN,COMMIT,andABORTqueriestotheDBMS.SincemanySQLoperatorsbehavedifferentlyonNULLsthanonnon-NULLvalues,CryptDBexposesNULLvaluestotheDBMSwithoutencryption.CryptDBdoesnotcurrentlysupportstoredprocedures,althoughcertainstoredprocedurescouldbesupportedbyrewritingtheircodeinthesamewaythatCryptDB'sproxyrewritesSQLstatements.TheDBMSbuildsindexesforencrypteddatainthesamewayasforplaintext.Currently,iftheapplicationrequestsanindexonacolumn,theproxyaskstheDBMSservertobuildindexesonthatcolumn'sDET,JOIN,OPE,orOPE-JOINonionlayers(iftheyareexposed),butnotforRND,HOM,orSEARCH.Moreefcientindexselectionalgorithmscouldbeinvestigated.3.4ComputingJoinsTherearetwokindsofjoinssupportedbyCryptDB:equi-joins,inwhichthejoinpredicateisbasedonequality,andrangejoins,whichinvolveorderchecks.Toperformanequi-joinoftwoencryptedcolumns,thecolumnsshouldbeencryptedwiththesamekeysothattheservercanseematchingvaluesbetweenthetwocolumns.Atthesametime,toprovidebetterprivacy,theDBMSservershouldnotbeabletojoincolumnsforwhichtheapplicationdidnotrequestajoin,socolumnsthatareneverjoinedshouldnotbeencryptedwiththesamekeys.Ifthequeriesthatcanbeissued,orthepairsofcolumnsthatcanbejoined,areknownapriori,equi-joiniseasytosupport:CryptDB6 3.5.2PerformanceOptimizationsDeveloperannotations.Bydefault,CryptDBencryptsalleldsandcreatesallapplicableonionsforeachdataitembasedonitstype.Ifmanycolumnsarenotsensitive,thedevelopercaninsteadprovideexplicitannotationsindicatingthesensitiveelds(asdescribedinx4),andleavetheremainingeldsinplaintext.Knownqueryset.Ifthedeveloperknowssomeofthequeriesaheadoftime,asisthecaseformanywebapplications,thedevelopercanusethetrainingmodedescribedabovetoadjustonionstothecorrectlayerapriori,avoidingtheoverheadofruntimeonionadjust-ments.Ifthedeveloperprovidestheexactqueryset,orannotationsthatcertainfunctionalityisnotneededonsomecolumns,CryptDBcanalsodiscardonionsthatarenotneeded(e.g.,discardtheOrdonionforcolumnsthatarenotusedinrangequeries,ordiscardtheSearchonionforcolumnswherekeywordsearchisnotperformed),discardonionlayersthatarenotneeded(e.g.,theadjustableJOINlayer,ifjoinsareknownapriori),ordiscardtherandomIVneededforRNDforsomecolumns.Ciphertextpre-computingandcaching.Theproxyspendsasig-nicantamountoftimeencryptingvaluesusedinquerieswithOPEandHOM.Toreducethiscost,theproxypre-computes(forHOM)andcaches(forOPE)encryptionsoffrequentlyusedconstantsunderdifferentkeys.SinceHOMisprobabilistic,ciphertextscannotbereused.Therefore,inaddition,theproxypre-computesHOM'sPail-lierrnrandomnessvaluesforfutureencryptionsofanydata.ThisoptimizationreducestheamountofCPUtimespentbytheproxyonOPEencryption,andassumingtheproxyisoccasionallyidletoperformHOMpre-computation,itremovesHOMencryptionfromthecriticalpath.4MULTIPLEPRINCIPALSWenowextendthethreatmodeltothecasewhentheapplicationinfrastructureandproxyarealsountrusted(threat2).Thismodelisespeciallyrelevantforamulti-userwebsiterunningawebandapplicationserver.Tounderstandboththeproblemsfacedbyamulti-userwebapplicationandCryptDB'ssolutiontotheseproblems,considerphpBB,apopularonlinewebforum.InphpBB,eachuserhasanaccountandapassword,belongstocertaingroups,andcansendprivatemessagestootherusers.Dependingontheirgroups'permissions,userscanreadentireforums,onlyforumnames,ornotbeabletoreadaforumatall.ThereareseveralcondentialityguaranteesthatwouldbeusefulinphpBB.Forexample,wewouldliketoensurethataprivatemessagesentfromoneusertoanotherisnotvisibletoanyoneelse;thatpostsinaforumareaccessibleonlytousersinagroupwithaccesstothatforum;andthatthenameofaforumisshownonlytousersbelongingtoagroupthat'sallowedtoviewit.CryptDBprovidestheseguaranteesinthefaceofarbitrarycompromises,therebylimitingthedamagecausedbyacompromise.Achievingtheseguaranteesrequiresaddressingtwochallenges.First,CryptDBmustcapturetheapplication'saccesscontrolpolicyforshareddataatthelevelofSQLqueries.Todothis,CryptDBrequiresdeveloperstoannotatetheirdatabaseschematospecifyprincipalsandthedatathateachprincipalhasaccessto,asdescribedinx4.1.Thesecondchallengeistoreducetheamountofinformationthatanadversarycangainbycompromisingthesystem.Oursolutionlimitstheleakageresultingfromacompromisedapplicationorproxyservertojustthedataaccessibletouserswhowereloggedinduringthecompromise.Inparticular,theattackercannotaccessthedataofusersthatwerenotloggedinduringthecompromise.Leakingthedataofactiveusersincaseofacompromiseisunavoidable:giventheimpracticalityofarbitrarycomputationonencrypteddata,somedataforactiveusersmustbedecryptedbytheapplication.InCryptDB,eachuserhasakey(e.g.,herapplication-levelpass-word)thatgivesheraccesstoherdata.CryptDBencryptsdifferentdataitemswithdifferentkeys,andenforcestheaccesscontrolpolicyusingchainsofkeysstartingfromuserpasswordsandendingintheencryptionkeysofSQLdataitems,asdescribedinx4.2.Whenauserlogsin,sheprovidesherpasswordtotheproxy(viatheapplica-tion).Theproxyusesthispasswordtoderiveonionkeystoprocessqueriesonencrypteddata,aspresentedintheprevioussection,andtodecrypttheresults.Theproxycandecryptonlythedatathattheuserhasaccessto,basedontheaccesscontrolpolicy.Theproxygivesthedecrypteddatatotheapplication,whichcannowcomputeonit.Whentheuserlogsout,theproxydeletestheuser'skey.4.1PolicyAnnotationsToexpressthedataprivacypolicyofadatabase-backedapplicationatthelevelofSQLqueries,theapplicationdevelopercanannotatetheschemaofadatabaseinCryptDBbyspecifying,foranysubsetofdataitems,whichprincipalhasaccesstoit.Aprincipalisanentity,suchasauseroragroup,overwhichitisnaturaltospecifyanaccesspolicy.EachSQLqueryinvolvinganannotateddataitemrequirestheprivilegeofthecorrespondingprincipal.CryptDBdenesitsownnotionofprincipalsinsteadofusingexistingDBMSprincipalsfortworeasons:rst,manyapplicationsdonotmapapplication-leveluserstoDBMSprincipalsinasufcientlyne-grainedmanner,andsecond,CryptDBrequiresexplicitdelegationofprivilegesbetweenprincipalsthatisdifculttoextractinanautomatedwayfromanaccesscontrollistspecication.AnapplicationdeveloperannotatestheschemausingthethreestepsdescribedbelowandillustratedinFigure4.Inallexamplesweshow,italicsindicatetableandcolumnnames,andboldtextindicatesannotationsaddedforCryptDB.Step1.Thedevelopermustdenetheprincipaltypes(usingPRINCTYPE)usedinherapplication,suchasusers,groups,ormes-sages.Aprincipalisaninstanceofaprincipaltype,e.g.,principal5oftypeuser.Therearetwoclassesofprincipals:externalandinternal.Externalprincipalscorrespondtoenduserswhoexplicitlyauthenticatethemselvestotheapplicationusingapassword.Whenauserlogsintotheapplication,theapplicationmustprovidetheuserpasswordtotheproxysothattheusercangettheprivilegesofherexternalprincipal.Privilegesofother(internal)principalscanbeacquiredonlythroughdelegation,asdescribedinStep3.Whentheuserlogsout,theapplicationmustinformtheproxy,sothattheproxyforgetstheuser'spasswordaswellasanykeysderivedfromtheuser'spassword.Step2.ThedevelopermustspecifywhichcolumnsinherSQLschemacontainsensitivedata,alongwiththeprincipalsthatshouldhaveaccesstothatdata,usingtheENC FORannotation.CryptDBrequiresthatforeachprivatedataiteminarow,thenameoftheprincipalthatshouldhaveaccesstothatdatabestoredinanothercolumninthesamerow.Forexample,inFigure4,thedecryptionofmsgtextx37a21fisavailableonlytoprincipal5oftypemsg.Step3.Programmerscanspecifyrulesforhowtodelegatetheprivilegesofoneprincipaltootherprincipals,usingthespeaks-forrelation[49].Forexample,inphpBB,ausershouldalsohavetheprivilegesofthegroupsshebelongsto.Sincemanyapplica-tionsstoresuchinformationintables,programmerscanspecifytoCryptDBhowtoinferdelegationrulesfromrowsinanexistingtable.Inparticular,programmerscanannotateatableTwith(ax)SPEAKS FOR(by).Thisannotationindicatesthateachrowpresentinthattablespeciesthatprincipalaoftypexspeaksfor8 PRINCTYPEphysical userEXTERNAL;PRINCTYPEuser,msg;CREATETABLEprivmsgs(msgidint,subjectvarchar(255)ENC FOR(msgidmsg),msgtexttextENC FOR(msgidmsg));CREATETABLEprivmsgs to(msgidint,rcpt idint,sender idint,(sender iduser)SPEAKS FOR(msgidmsg),(rcpt iduser)SPEAKS FOR(msgidmsg));CREATETABLEusers(useridint,usernamevarchar(255),(usernamephysical user)SPEAKS FOR(useriduser)); Exampletablecontents,withoutanonymizedcolumnnames:Tableprivmsgsmsgid subject msgtext 5 xcc82fa x37a21fTableprivmsgs tomsgid rcpt id sender id 5 1 2Tableusersuserid username 1 `Alice'2 `Bob' Figure4:PartofphpBB'sschemawithannotationstosecureprivatemessages.Onlythesenderandreceivermayseetheprivatemessage.AnattackerthatgainscompleteaccesstophpBBandtheDBMScanaccessprivatemessagesofonlycurrentlyactiveusers.principalboftypey,meaningthatahasaccesstoallkeysthatbhasaccessto.Here,xandymustalwaysbexedprincipaltypes.Princi-palbisalwaysspeciedbythenameofacolumnintableT.Ontheotherhand,acanbeeitherthenameofanothercolumninthesametable,aconstant,orT2:col,meaningallprincipalsfromcolumncoloftableT2.Forexample,inFigure4,principal“Bob”oftypephysical userspeaksforprincipal2oftypeuser,andinFigure6,allprincipalsinthecontactIdcolumnfromtablePCMember(oftypecontact)speakforthepaperIdprincipaloftypereview.Optionally,theprogrammercanspecifyapredicate,whoseinputsarevaluesinthesamerow,tospecifyaconditionunderwhichdelegationshouldoccur,suchasexcludingconictsinFigure6.x5providesmoreexamplesofusingannotationstosecureapplications.4.2KeyChainingEachprincipal(i.e.,eachinstanceofeachprincipaltype)isasso-ciatedwithasecret,randomlychosenkey.IfprincipalBspeaksforprincipalA(asaresultofsomeSPEAKS FORannotation),thenprincipalA'skeyisencryptedusingprincipalB'skey,andstoredasarowinthespecialaccess keystableinthedatabase.ThisallowsprincipalBtogainaccesstoprincipalA'skey.Forexample,inFigure4,togiveusers1and2accesstomessage5,thekeyofmsg5isencryptedwiththekeyofuser1,andalsoseparatelyencryptedwiththekeyofuser2.EachsensitiveeldisencryptedwiththekeyoftheprincipalintheENC FORannotation.CryptDBencryptsthesensitiveeldwithonionsinthesamewayasforsingle-principalCryptDB,exceptthatonionkeysarederivedfromaprincipal'skeyasopposedtoaglobalmasterkey.Thekeyofeachprincipalisacombinationofasymmetrickeyandapublic–privatekeypair.Inthecommoncase,CryptDBusesthesymmetrickeyofaprincipaltoencryptanydataandotherprincipals'keysaccessibletothisprincipal,withlittleCPUcost.However,thisisnotalwayspossible,ifsomeprincipalisnotcurrentlyonline.Forexample,inFigure4,supposeBobsendsmessage5toAlice,butAlice(user1)isnotonline.ThismeansthatCryptDBdoesnothaveaccesstouser1'skey,soitwillnotbeabletoencryptmessage5'skeywithuser1'ssymmetrickey.Inthiscase,CryptDBlooksupthepublickeyoftheprincipal(i.e.,user1)inasecondtable,public keys,andencryptsmessage5'skeyusinguser1'spublickey.Whenuser1logsin,shewillbeabletousethesecretkeypartofherkeytodecryptthekeyformessage5(andre-encryptitunderhersymmetrickeyforfutureuse).Forexternalprincipals(i.e.,physicalusers),CryptDBassignsarandomkeyjustasforanyotherprincipal.Togiveanexternaluseraccesstothecorrespondingkeyonlogin,CryptDBstoresthekeyofeachexternalprincipalinathirdtable,external keys,encryptedwiththeprincipal'spassword.ThisallowsCryptDBtoobtainauser'skeygiventheuser'spassword,andalsoallowsausertochangeherpasswordwithoutchangingthekeyoftheprincipal.WhenatablewithaSPEAKS FORrelationisupdated,CryptDBmustupdatetheaccess keystableaccordingly.Toinsertanewrowintoaccess keysforanewSPEAKS FORrelation,theproxymusthaveaccesstothekeyoftheprincipalwhoseprivilegesarebeingdelegated.ThismeansthatanadversarythatbreaksintoanapplicationorproxyservercannotcreatenewSPEAKS FORrelationsforprincipalsthatarenotloggedin,becauseneithertheproxynortheadversaryhaveaccesstotheirkeys.IfaSPEAKS FORrelationisremoved,CryptDBrevokesaccessbyremovingthecorrespondingrowfromaccess keys.Whenencryptingdatainaqueryordecryptingdatafromaresult,CryptDBfollowskeychainsstartingfrompasswordsofusersloggedinuntilitobtainsthedesiredkeys.Asanoptimization,whenauserlogsin,CryptDB'sproxyloadsthekeysofsomeprincipalstowhichtheuserhasaccess(inparticular,thoseprincipaltypesthatdonothavetoomanyprincipalinstances—e.g.,forgroupstheuserisin,butnotformessagestheuserreceived).ApplicationsinformCryptDBofuserslogginginoroutbyissuingINSERTandDELETESQLqueriestoaspecialtablecryptdb activethathastwocolumns,usernameandpassword.Theproxyinterceptsallqueriesforcryptdb active,storesthepasswordsoflogged-inusersinmemory,andneverrevealsthemtotheDBMSserver.CryptDBguardsthedataofinactiveusersatthetimeofanattack.Ifacompromiseoccurs,CryptDBprovidesaboundonthedataleaked,allowingtheadministratorstonotissueablanketwarningtoalltheusersofthesystem.Inthisrespect,CryptDBisdifferentfromotherapproachestodatabasesecurity(seex9).However,somespecialuserssuchasadministratorswithaccesstoalargepoolofdataenablealargercompromiseuponanattack.Toavoidattackshappeningwhentheadministratorisloggedin,theadministratorshouldcreateaseparateuseraccountwithrestrictedpermissionswhenaccessingtheapplicationasaregularuser.Also,asgoodpractice,anapplicationshouldautomaticallylogoutuserswhohavebeeninactiveforsomeperiodoftime.5APPLICATIONCASESTUDIESInthissection,weexplainhowCryptDBcanbeusedtosecurethreeexistingmulti-userwebapplications.Forbrevity,weshowsimpliedschemas,omittingirrelevanteldsandtypespeciers.Overall,wendthatonceaprogrammerspeciestheprincipalsintheapplication'sschema,andthedelegationrulesforthemus-ingSPEAKS FOR,protectingadditionalsensitiveeldsjustrequiresadditionalENC FORannotations.phpBBisawidelyusedopensourceforumwitharichsetofaccesscontrolsettings.Usersareorganizedingroups;bothusersandgroupshaveavarietyofaccesspermissionsthattheapplication9 PRINCTYPEphysical userEXTERNAL;PRINCTYPEuser,group,forum post,forum name;CREATETABLEusers(useridint,usernamevarchar(255),(usernamephysical user)SPEAKS FOR(useriduser));CREATETABLEusergroup(useridint,groupidint,(useriduser)SPEAKS FOR(groupidgroup));CREATETABLEaclgroups(groupidint,forumidint,optionidint,(groupidgroup)SPEAKS FOR(forumidforum post)IFoptionid=20,(groupidgroup)SPEAKS FOR(forumidforum name)IFoptionid=14);CREATETABLEposts(postidint,forumidint,posttextENC FOR(forumidforum post));CREATETABLEforum(forumidint,namevarchar(255)ENC FOR(forumidforum name)); Figure5:AnnotatedschemaforsecuringaccesstopostsinphpBB.Auserhasaccesstoseethecontentofpostsinaforumifanyofthegroupsthattheuserispartofhassuchpermissions,indicatedbyoptionid20intheaclgroupstableforthecorrespondingforumidandgroupid.Similarly,optionid14enablesuserstoseetheforum'sname.administratorcanchoose.WealreadyshowedhowtosecureprivatemessagesbetweentwousersinphpBBinFigure4.Amoredetailedcaseissecuringaccesstoposts,asshowninFigure5.Thisexampleshowshowtousepredicates(e.g.,IFoptionid=...)toimple-mentaconditionalspeaks-forrelationonprincipals,andalsohowonecolumn(forumid)canbeusedtorepresentmultipleprincipals(ofdifferenttype)withdifferentprivileges.Therearemorewaystogainaccesstoapost,butweomitthemhereforbrevity.HotCRPisapopularconferencereviewapplication[27].AkeypolicyforHotCRPisthatPCmemberscannotseewhoreviewedtheirown(orconicted)papers.Figure6showsCryptDBannota-tionsforHotCRP'sschematoenforcethispolicy.Today,HotCRPcannotpreventacuriousorcarelessPCchairfromloggingintothedatabaseserverandseeingwhowroteeachreviewforapaperthatsheisinconictwith.Asaresult,conferencesoftensetupasecondservertoreviewthechair'spapersoruseinconvenientout-of-bandemails.WithCryptDB,aPCchaircannotlearnwhowroteeachreviewforherpaper,evenifshebreaksintotheapplicationordatabase,sinceshedoesnothavethedecryptionkey.1ThereasonisthattheSQLpredicate“NoConict”checksifaPCmemberisconictedwithapaperandpreventstheproxyfromprovidingaccesstothePCchairinthekeychain.(WeassumethePCchairdoesnotmodifytheapplicationtologthepasswordsofotherPCmemberstosubvertthesystem.)grad-applyisagraduateadmissionssystemusedbyMITEECS.Weannotateditsschematoallowanapplicant'sfoldertobeaccessedonlybytherespectiveapplicantandanyfacultyus-ing(reviewers.reviewer idreviewer),meaningallreview-ers,SPEAKS FOR(candidate idcandidate)intablecandi-dates,and...SPEAKS FOR(letter idletter)intablelet-ters.Theapplicantcanseeallofherfolderdataexceptforlettersofrecommendation.Overall,grad-applyhassimpleaccesscontrolandthereforesimpleannotations. 1FullyimplementingthispolicywouldrequiresettinguptwoPCchairs:amainchair,andabackupchairresponsibleforreviewsofthemainchair'spapers.HotCRPallowsthePCchairtoimpersonateotherPCmembers,soCryptDBannotationswouldbeusedtopreventthemainchairfromgainingaccesstokeysofreviewersassignedtoherpaper. PRINCTYPEphysical userEXTERNAL;PRINCTYPEcontact,review;CREATETABLEContactInfo(contactIdint,emailvarchar(120),(emailphysical user)SPEAKS FOR(contactIdcontact));CREATETABLEPCMember(contactIdint);CREATETABLEPaperConict(paperIdint,contactIdint);CREATETABLEPaperReview(paperIdint,reviewerIdintENC FOR(paperIdreview),commentsToPCtextENC FOR(paperIdreview),(PCMember.contactIdcontact)SPEAKS FOR(paperIdreview)IFNoConict(paperId,contactId));NoConict(paperId,contactId):/*DeneaSQLfunction*/(SELECTCOUNT(*)FROMPaperConictcWHEREc.paperId=paperIdANDc.contactId=contactId)=0; Figure6:AnnotatedschemaforsecuringreviewsinHotCRP.ReviewsandtheidentityofreviewersprovidingthereviewwillbeavailableonlytoPCmembers(tablePCMemberincludesPCchairs)whoarenotconicted,andPCchairscannotoverridethisrestriction.6DISCUSSIONCryptDB'sdesignsupportsmostrelationalqueriesandaggregatesonstandarddatatypes,suchasintegersandtext/varchartypes.Addi-tionaloperationscanbeaddedtoCryptDBbyextendingitsexistingonions,oraddingnewonionsforspecicdatatypes(e.g.,spatialandmulti-dimensionalrangequeries[43]).Alternatively,insomecases,itmaybepossibletomapcomplexunsupportedoperationtosimplerones(e.g.,extractingthemonthoutofanencrypteddateiseasierifthedate'sday,month,andyeareldsareencryptedseparately).TherearecertaincomputationsCryptDBcannotsupportonen-crypteddata.Forexample,itdoesnotsupportbothcomputationandcomparisononthesamecolumn,suchasWHEREsalary�age*2+10.CryptDBcanprocessapartofthisquery,butitwouldalsorequiresomeprocessingontheproxy.InCryptDB,suchaqueryshouldbe(1)rewrittenintoasub-querythatselectsawholecolumn,SELECTage*2+10FROM:::,whichCryptDBcomputesusingHOM,and(2)re-encryptedintheproxy,creatinganewcol-umn(callitaux)ontheDBMSserverconsistingofthenewlyen-cryptedvalues.Finally,theoriginalquerywiththepredicateWHEREsalary�auxshouldberun.Wehavenotbeenaffectedbythislimitationinourtestapplications(TPC-C,phpBB,HotCRP,andgrad-apply).Inmulti-principalmode,CryptDBcannotperformserver-sidecomputationsonvaluesencryptedfordifferentprincipals,eveniftheapplicationhastheauthorityofallprincipalsinquestion,be-causetheciphertextsareencryptedwithdifferentkeys.Forsomecomputations,itmaybepracticalfortheproxytoperformthecom-putationafterdecryptingthedata,butforothers(e.g.,large-scaleaggregates)thisapproachmaybetooexpensive.Apossibleexten-siontoCryptDBtosupportsuchqueriesmaybetomaintainmultipleciphertextsforsuchvalues,encryptedunderdifferentkeys.7IMPLEMENTATIONTheCryptDBproxyconsistsofaC++libraryandaLuamodule.TheC++libraryconsistsofaqueryparser;aqueryencryptor/rewriter,whichencryptseldsorincludesUDFsinthequery;andare-sultdecryptionmodule.ToallowapplicationstotransparentlyuseCryptDB,weusedMySQLproxy[47]andimplementedaLuamod-ulethatpassesqueriesandresultstoandfromourC++module.WeimplementedournewcryptographicprotocolsusingNTL[44].Our10 Application AnnotationsLogin/logoutcodeSensitiveeldssecured,andexamplesofsuchelds phpBB 31(11unique)7lines23:privatemessages(content,subject),posts,forumsHotCRP 29(12unique)2lines22:papercontentandpaperinformation,reviewsgrad-apply 111(13unique)2lines103:studentgrades(61),scores(17),recommendations,reviewsTPC-C(singleprinc.) 0092:alltheeldsinallthetablesencryptedFigure8:Numberofannotationstheprogrammerneedstoaddtosecuresensitiveelds,linesofcodetobeaddedtoprovideCryptDBwiththepasswordsofusers,andthenumberofsensitiveeldsthatCryptDBsecureswiththeseannotations,forthreedifferentapplications.WecountasoneannotationeachinvocationofourthreetypesofannotationsandanySQLpredicateusedinaSPEAKS FORannotation.Sincemultipleeldsinthesametableareusuallyencryptedforthesameprincipal(e.g.,messagesubjectandcontent),wealsoreportuniqueannotations.Application TotalConsiderNeedsNeedsNeeds Non-plaintextcols.withMinEnc: Mostsensitive cols.forenc.plaintextHOMSEARCH RNDSEARCHDETOPE cols.atHIGH phpBB 56323010 21011 6/6HotCRP 20422021 18112 18/18grad-apply 706103002 95062 94/94OpenEMR 1;297566703 52621219 525/540MIT6.02 1513000 7042 1/1PHP-calendar 2512202 3241 3/4 TPC-C 9292080 650198 —Tracefromsql.mit.edu 128;840128;8401;0941;0191;125 80;05335034;21213;131 —:::within-proxyprocessing 128;840128;8405711;0161;135 84;00839835;3508;513 —:::col.namecontainspass 2;0292;029200 1;9360910 —:::col.namecontainscontent 2;5212;5210052 2;215522513 —:::col.namecontainspriv 173173040 1590122 —Figure9:Steady-stateonionlevelsfordatabasecolumnsrequiredbyarangeofapplicationsandtraces.“Needsplaintext”indicatesthatCryptDBcannotexecutetheapplication'squeriesoverencrypteddataforthatcolumn.Fortheapplicationsinthetopgroupofrows,sensitivecolumnsweredeterminedmanually,andonlythesecolumnswereconsideredforencryption.Forthebottomgroupofrows,alldatabasecolumnswereautomaticallyconsideredforencryption.Therightmostcolumnconsiderstheapplication'smostsensitivedatabasecolumns,andreportsthenumberofthemthathaveMinEncinHIGH(bothtermsaredenedinx8.3).quantifythelevelofsecurity,wedenetheMinEncofacolumntobetheweakestonionencryptionschemeexposedonanyoftheonionsofacolumnwhenonionsreachasteadystate(i.e.,aftertheapplicationgeneratesallquerytypes,orafterrunningthewholetrace).WeconsiderRNDandHOMtobethestrongestschemes,followedbySEARCH,followedbyDETandJOIN,andnishingwiththeweakestschemewhichisOPE.Forexample,ifacolumnhasonionEqatRND,onionOrdatOPEandonionAddatHOM,theMinEncofthiscolumnisOPE.TherightsideofFigure9showstheMinEnconionlevelforarangeofapplicationsandquerytraces.WeseethatmosteldsremainatRND,whichisthemostsecurescheme.Forexample,OpenEMRhashundredsofsensitiveeldsdescribingthemedicalconditionsandhistoryofpatients,buttheseeldsaremostlyjustinsertedandfetched,andarenotusedinanycomputation.Anum-berofeldsalsoremainatDET,typicallytoperformkeylookupsandjoins.OPE,whichleaksorder,isusedtheleastfrequently,andmostlyforeldsthataremarginallysensitive(e.g.,timestampsandcountsofmessages).Thus,CryptDB'sadjustablesecuritypro-videsasignicantimprovementincondentialityoverrevealingallencryptionschemestotheserver.ToanalyzeCryptDB'ssecurityforspeciccolumnsthatarepar-ticularlysensitive,wedeneanewsecuritylevel,HIGH,whichincludestheRNDandHOMencryptionschemes,aswellasDETforcolumnshavingnorepetitions(inwhichcaseDETislogicallyequivalenttoRND).Thesearehighlysecureencryptionschemesleakingvirtuallynothingaboutthedata.DETforcolumnswithrepeatsandOPEarenotpartofHIGHastheyrevealrelationstotheDBMSserver.TherightmostcolumninFigure9showsthatmostoftheparticularlysensitivecolumns(again,accordingtomanualinspection)areatHIGH.Forthesql.mit.edutracequeries,approximately6.6%ofcolumnswereatOPEevenwithin-proxyprocessing;otheren-cryptedcolumns(93%)remainatDETorabove.OutofthecolumnsthatwereatOPE,3.9%areusedinanORDERBYclausewithaLIMIT,3.7%areusedinaninequalitycomparisoninaWHEREclause,and0.25%areusedinaMINorMAXaggregateoperator(someofthecolumnsarecountedinmorethanoneofthesegroups).Itwouldbedifculttoperformthesecomputationsintheproxywithoutsubstantiallyincreasingtheamountofdatasenttoit.Althoughwecouldnotexaminetheschemasofapplicationsus-ingsql.mit.edutodeterminewhateldsaresensitive—mostlyduetoitslargescale—wemeasuredthesamestatisticsasaboveforcolumnswhosenamesareindicativeofsensitivedata.Inparticular,thelastthreerowsofFigure9showcolumnswhosenamecontainstheword“pass”(whicharealmostallsometypeofpassword),“con-tent”(whicharetypicallybulkdatamanagedbyanapplication),and“priv”(whicharetypicallysometypeofprivatemessage).CryptDBrevealsmuchlessinformationaboutthesecolumnsthananaveragecolumn,almostallofthemaresupported,andalmostallareatRNDorDET.Finally,weempiricallyvalidatedCryptDB'scondentialityguar-anteesbytryingrealattacksonphpBBthathavebeenlistedintheCVEdatabase[32],includingtwoSQLinjectionattacks(CVE-2009-3052&CVE-2008-6314),bugsinpermissionchecks(CVE-2010-1627&CVE-2008-7143),andabuginremotePHPleinclusion(CVE-2008-6377).Wefoundthat,forusersnotcurrentlyloggedin,theanswersreturnedfromtheDBMSwereencrypted;evenwithrootaccesstotheapplicationserver,proxy,andDBMS,theanswerswerenotdecryptable.8.4PerformanceEvaluationToevaluatetheperformanceofCryptDB,weusedamachinewithtwo2.4GHzIntelXeonE56204-coreprocessorsand12GBofRAMtoruntheMySQL5.1.54server,andamachinewitheight2.4GHzAMDOpteron84316-coreprocessorsand64GBofRAMtoruntheCryptDBproxyandtheclients.ThetwomachineswereconnectedoverasharedGigabitEthernetnetwork.Thehigher-provisionedclientmachineensuresthattheclientsarenotthebottleneckinanyexperiment.Allworkloadstintheserver'sRAM.12 Figure10:ThroughputforTPC-Cqueries,foravaryingnumberofcoresontheunderlyingMySQLDBMSserver. Figure11:ThroughputofdifferenttypesofSQLqueriesfromtheTPC-CquerymixrunningunderMySQL,CryptDB,andthestrawmandesign.“Upd.inc”standsforUPDATEthatincrementsacolumn,and“Upd.set”standsforUPDATEwhichsetscolumnstoaconstant.8.4.1TPC-CWecomparetheperformanceofaTPC-CquerymixwhenrunningonanunmodiedMySQLserverversusonaCryptDBproxyinfrontoftheMySQLserver.WetrainedCryptDBonthequeryset(x3.5.2)sotherearenoonionadjustmentsduringtheTPC-Cexperiments.Figure10showsthethroughputofTPC-Cqueriesasthenumberofcoresontheservervariesfromonetoeight.Inallcases,theserverspends100%ofitsCPUtimeprocessingqueries.BothMySQLandCryptDBscalewellinitially,butstarttoleveloffduetointernallockcontentionintheMySQLserver,asreportedbySHOWSTATUSLIKE'Table%'.TheoverallthroughputwithCryptDBis21–26%lowerthanMySQL,dependingontheexactnumberofcores.TounderstandthesourcesofCryptDB'soverhead,wemeasuretheserverthroughputfordifferenttypesofSQLqueriesseeninTPC-C,onthesameserver,butrunningwithonlyonecoreenabled.Figure11showstheresultsforMySQL,CryptDB,andastrawmandesign;thestrawmanperformseachqueryoverdataencryptedwithRNDbydecryptingtherelevantdatausingaUDF,performingthequeryovertheplaintext,andre-encryptingtheresult(ifupdatingrows).TheresultsshowthatCryptDB'sthroughputpenaltyisgreat-estforqueriesthatinvolveaSUM(2.0lessthroughput)andforincrementingUPDATEstatements(1.6lessthroughput);thesearethequeriesthatinvolveHOMadditionsattheserver.Fortheothertypesofqueries,whichformalargerpartoftheTPC-Cmix,thethroughputoverheadismodest.ThestrawmandesignperformspoorlyforalmostallqueriesbecausetheDBMS'sindexesontheQuery(&scheme) MySQL CryptDB Server Server Proxy Proxy? Selectby=(DET) 0:10ms 0:11ms 0:86ms 0:86msSelectjoin(JOIN) 0:10ms 0:11ms 0:75ms 0:75msSelectrange(OPE) 0:16ms 0:22ms 0:78ms 28:7msSelectsum(HOM) 0:11ms 0:46ms 0:99ms 0:99msDelete 0:07ms 0:08ms 0:28ms 0:28msInsert(all) 0:08ms 0:10ms 0:37ms 16:3msUpdateset(all) 0:11ms 0:14ms 0:36ms 3:80msUpdateinc(HOM) 0:10ms 0:17ms 0:30ms 25:1ms Overall 0:10ms 0:12ms 0:60ms 10:7msFigure12:ServerandproxylatencyfordifferenttypesofSQLqueriesfromTPC-C.Foreachquerytype,weshowthepredominantencryptionschemeusedattheserver.DuetodetailsoftheTPC-Cworkload,eachquerytypeaffectsadifferentnumberofrows,andinvolvesadifferentnumberofcryptographicoperations.Thelefttwocolumnscorrespondtoserverthroughput,whichisalsoshowninFigure11.“Proxy”showsthelatencyaddedbyCryptDB'sproxy;“Proxy?”showstheproxylatencywithouttheciphertextpre-computingandcachingoptimization(x3.5).Boldnumbersshowwherepre-computingandcachingciphertextshelps.The“Overall”rowistheaveragelatencyoverthemixofTPC-Cqueries.“Updateset”isanUPDATEwheretheeldsaresettoaconstant,and“Updateinc”isanUPDATEwheresomeeldsareincremented.Scheme EncryptDecryptSpecialoperation Blowsh(1int.) 0.0001ms0.0001ms—AES-CBC(1KB) 0.008ms0.007ms—AES-CMC(1KB) 0.016ms0.015ms—OPE(1int.) 9.0ms9.0msCompare:0msSEARCH(1word) 0.01ms0.004msMatch:0.001msHOM(1int.) 9.7ms0.7msAdd:0.005msJOIN-ADJ(1int.) 0.52ms—Adjust:0.56msFigure13:Microbenchmarksofcryptographicschemes,perunitofdataencrypted(one32-bitinteger,1KB,orone15-bytewordoftext),measuredbytakingtheaveragetimeovermanyiterations.RND-encrypteddataareuselessforoperationsontheunderlyingplaintextdata.ItispleasantlysurprisingthatthehighersecurityofCryptDBoverthestrawmanalsobringsbetterperformance.TounderstandthelatencyintroducedbyCryptDB'sproxy,wemeasuretheserverandproxyprocessingtimesforthesametypesofSQLqueriesasabove.Figure12showstheresults.Wecanseethatthereisanoverallserverlatencyincreaseof20%withCryptDB,whichweconsidermodest.Theproxyaddsanaverageof0.60mstoaquery;ofthattime,24%isspentinMySQLproxy,23%isspentinencryptionanddecryption,andtheremaining53%isspentparsingandprocessingqueries.Thecryptographicoverheadisrelativelysmallbecausemostofourencryptionschemesareefcient;Figure13showstheirperformance.OPEandHOMaretheslowest,buttheciphertextpre-computingandcachingoptimization(x3.5)masksthehighlatencyofqueriesrequiringOPEandHOM.Proxy?inFigure12showsthelatencywithouttheseoptimizations,whichissignicantlyhigherforthecorrespondingquerytypes.SELECTqueriesthatinvolveaSUMuseHOMbutdonotbenetfromthisoptimization,becausetheproxyperformsdecryption,ratherthanencryption.InallTPC-Cexperiments,theproxyusedlessthan20MBofmemory.Cachingciphertextsforthe30;000mostcommonvaluesforOPEaccountsforabout3MB,andpre-computingciphertextsandrandomnessfor30,000valuesatHOMrequired10MB.8.4.2Multi-UserWebApplicationsToevaluatetheimpactofCryptDBonapplicationperformance,wemeasurethethroughputofphpBBforaworkloadwith10parallelclients,whichensured100%CPUloadattheserver.EachclientcontinuouslyissuedHTTPrequeststobrowsetheforum,writeand13 Figure14:ThroughputcomparisonforphpBB.“MySQL”denotesphpBBrunningdirectlyonMySQL.“MySQL+proxy”denotesphpBBrunningonanunencryptedMySQLdatabasebutgoingthroughMySQLproxy.“CryptDB”denotesphpBBrunningonCryptDBwithnotablysensitiveeldsannotatedandthedatabaseappropriatelyencrypted.MostHTTPrequestsinvolvedtensofSQLquerieseach.PercentagesindicatethroughputreductionrelativetoMySQL.DB LoginRpostWpostRmsgWmsg MySQL 60ms50ms133ms61ms237msCryptDB 67ms60ms151ms73ms251msFigure15:LatencyforHTTPrequeststhatheavilyuseencryptedeldsinphpBBforMySQLandCryptDB.RandWstandforreadandwrite.readposts,aswellaswriteandreadprivatemessages.Wepre-loadedforumsandusermailboxeswithmessages.Inthisexperiment,weco-locatedtheMySQLDBMS,theCryptDBproxy,andthewebapplicationserveronasingle-coremachine,toensurewedonotaddadditionalresourcesforaseparateproxyservermachinetothesystemintheCryptDBconguration.Inpractice,anadministratorwouldlikelyruntheCryptDBproxyonanothermachineforsecurity.Figure14showsthethroughputofphpBBinthreedifferentcon-gurations:(1)connectingtoastockMySQLserver,(2)connectingtoastockMySQLserverthroughMySQLproxy,and(3)connectingtoCryptDB,withnotablysensitiveeldsencryptedassummarizedinFigure9,whichinturnusesastockMySQLservertostoreencrypteddata.TheresultsshowthatphpBBincursanoverallthroughputlossofjust14.5%,andthatabouthalfofthislosscomesfrominefcienciesinMySQLproxyunrelatedtoCryptDB.Fig-ure15furthershowstheend-to-endlatencyforvetypesofphpBBrequests.TheresultsshowthatCryptDBadds7–18ms(6–20%)ofprocessingtimeperrequest.8.4.3StorageCryptDBincreasestheamountofthedatastoredintheDBMS,becauseitstoresmultipleonionsforthesameeld,andbecauseciphertextsarelargerthanplaintextsforsomeencryptionschemes.ForTPC-C,CryptDBincreasedthedatabasesizeby3.76,mostlyduetocryptographicexpansionofintegereldsencryptedwithHOM(whichexpandfrom32bitsto2048bits);stringsandbinarydataremainsroughlythesamesize.ForphpBB,thedatabasesizeusinganunencryptedsystemwas2.6MBforaworkloadofabout1,000privatemessagesand1,000forumpostsgeneratedby10users.ThesameworkloadonCryptDBhadadatabaseof3.3MB,about1.2larger.Ofthe0.7MBincrease,230KBisforstorageofaccess keys,276KBisforpublic keysandexternal keys,and166KBisduetoexpansionofencryptedelds.8.4.4AdjustableEncryptionAdjustablequery-basedencryptioninvolvesdecryptingcolumnstolower-securityonionlevels.Fortunately,decryptionforthemore-secureonionlayers,suchasRND,isfast,andneedstobeperformedonlyoncepercolumnforthelifetimeofthesystem.2RemovingalayerofRNDrequiresAESdecryption,whichourexperimentalmachinecanperformat200MB/spercore.Thus,removinganonionlayerisbottleneckedbythespeedatwhichtheDBMSservercancopyacolumnfromdiskfordisk-bounddatabases.9RELATEDWORKSearchandqueriesoverencrypteddata.Songetal.[46]describecryptographictoolsforperformingkeywordsearchoverencrypteddata,whichweusetoimplementSEARCH.Amanatidisetal.[2]proposemethodsforexactsearchesthatdonotrequirescanningtheentiredatabaseandcouldbeusedtoprocesscertainrestrictedSQLqueries.Baoetal.[3]extendtheseencryptedsearchmethodstothemulti-usercase.Yangetal.[51]runselectionswithequalitypredicatesoverencrypteddata.EvdokimovandGuentherpresentmethodsforthesameselections,aswellasCartesianproductsandprojections[15].Agrawaletal.developastatisticalencodingthatpreservestheorderofnumericaldatainacolumn[1],butitdoesnothavesoundcryptographicproperties,unliketheschemeweuse[4].BonehandWatersshowpublic-keyschemesforcomparisons,subsetchecks,andconjunctionsofsuchqueriesoverencrypteddata[5],buttheseschemeshaveciphertextlengthsthatareexponentialinthelengthoftheplaintext,limitingtheirpracticalapplicability.WhenappliedtoprocessingSQLonencrypteddata,thesetech-niquessufferfromsomeofthefollowinglimitations:certainbasicqueriesarenotsupportedoraretooinefcient(especiallyjoinsandorderchecks),theyrequiresignicantclient-sidequeryprocessing,userseitherhavetobuildandmaintainindexesonthedataattheserverortoperformsequentialscansforeveryselection/search,andimplementingthesetechniquesrequiresunattractivechangestotheinnardsoftheDBMS.SomeresearchershavedevelopedprototypesystemsforsubsetsofSQL,buttheyprovidenocondentialityguarantees,requireasignicantDBMSrewrite,andrelyonclient-sideprocessing[9,12,22].Forexample,Hacigumusetal.[22]heuristicallysplitthedomainofpossiblevaluesforeachcolumnintopartitions,storingthepartitionnumberunencryptedforeachdataitem,andrelyonextensiveclient-sidelteringofqueryresults.Chowetal.[8]requiretrustedentitiesandtwonon-colludinguntrustedDBMSes.Untrustedservers.SUNDR[28]usescryptographytoprovideprivacyandintegrityinalesystemontopofanuntrustedleserver.UsingaSUNDR-likemodel,SPORC[16]andDepot[30]showhowtobuildlow-latencyapplications,runningmostlyontheclients,withouthavingtotrustaserver.However,existingserver-sideappli-cationsthatinvolveseparatedatabaseandapplicationserverscannotbeusedwiththesesystemsunlesstheyarerewrittenasdistributedclient-sideapplicationstoworkwithSPORCorDepot.Manyappli-cationsarenotamenabletosuchastructure.CompanieslikeNavajoSystemsandCiphercloudprovideatrustedapplication-levelproxythatinterceptsnetworktrafcbe-tweenclientsandcloud-hostedservers(e.g.,IMAP),andencryptssensitivedatastoredontheserver.Theseproductsappeartobreakupsensitivedata(speciedbyapplication-specicrules)intotokens(suchaswordsinastring),andencrypteachofthesetokensusinganorder-preservingencryptionscheme,whichallowstoken-levelsearchingandsorting.Incontrast,CryptDBsupportsarichersetofoperations(mostofSQL),revealsonlyrelationsforthenecessaryclassesofcomputationtotheserverbasedonthequeriesissuedbytheapplication,andallowschainingofencryptionkeystouserpasswords,torestrictdataleaksfromacompromisedproxy. 2Unlesstheadministratorperiodicallyre-encryptsdata/columns.14