2cs262aS14 Lecture02Today ID: 507583
Download Pdf The PPT/PDF document "Advanced Topics in Computer SystemsEnd-t..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Advanced TopLcs Ln Computer S\stemsEnd-to-End / S\stem REOectrLcaO EnJLneerLnJ and Computer ScLencesUnLversLt\ of CaOLfornLa BerNeOe\ cs262a-F1 /ecture-02 Computer Systems,VoO2 No. 4 November 14 pp2-2. DonaOd D. CKamberOLn Morton A. AstraKan MLcKaeO :. BOasJen SeOLnJer MarLo ScKNoOnLcN DonaOd R. SOut] ,rvLnJ /. TraLJer cs262a-F1 /ecture-02 TKe +ourJOass ModeO:aLst 02.11 Transport cs262a-F1 /ecture-02 cs262a-F1 /ecture-02•/a\er N ma\ dupOLcate Oa\er N-1 functLonaOLt\ –E.J. error recover\ to retransmLt Oost data•/a\ers ma\ need same LnformatLon–E.J. tLmestamps ma[Lmum transmLssLon unLt sL]e•/a\erLnJ can Kurt performance–E.J. KLdLnJ detaLOs about ZKat Ls reaOO\ JoLnJ on•Some Oa\ers are not aOZa\s cOeanO\ separated–,nter-Oa\er dependencLes for performance reasons–Some dependencLes Ln standards Keader cKecNsums•+eaders start to Jet reaOO\ bLJ–SometLmes Keader b\tes >> actuaO content cs262a-F1 /ecture-02•+uJeO\ LnfOuentLaO paper End-to-End ArJuments Ln b\ SaOt]er Reed and COarN –EndOess dLsputes about ZKat Lt means–Ever\one cLtes Lt as supportLnJ tKeLr posLtLon cs262a-F1 /ecture-02•Some t\pes of netZorN functLonaOLt\ can onO\ be correctO\ LmpOemented –ReOLabLOLt\ securLt\ etc•Because of tKLs end Kosts–Can satLsf\ tKe requLrement ZLtKout netZorN’s KeOp–:LOO/must do so sLnce can’t •TKerefore don’t Jo out of \our Za\ to LmpOement cs262a-F1 /ecture-02 •SoOutLon 1 maNe eacK step reOLabOe and tKen •SoOutLon 2 end-to-end cKecN and tr\ aJaLn Lf Host AHost B OK cs262a-F1 /ecture-02•SoOutLon 1 Ls –:Kat Kappens Lf memor\ Ls corrupted?–ReceLver Kas to do tKe cKecN an\Za\!•SoOutLon 2 Ls –FuOO functLonaOLt\ can be entLreO\ LmpOemented at appOLcatLon Oa\er ZLtK need for reOLabLOLt\ from OoZer Oa\ersIs there any need to implement reliability at lower layers?–:eOO Lt couOd be cs262a-F1 /ecture-02End-to-End PrLncLpOe,mpOementLnJ tKLs functLonaOLt\ Ln tKe netZorN•Doesn’t reduce Kost LmpOementatLon compOe[Lt\•Does Lncrease netZorN compOe[Lt\•ProbabO\ Lmposes deOa\ and overKead on aOO even Lf tKe\ don’t need functLonaOLt\•+oZever LmpOementLnJ Ln netZorN performance Ln some cases–E.J. ver\ Ooss\OLnNs sucK as ZLreOess•,t ma\ aOso KeOp mLtLJate denLaO of servLce and/or prLvac\ cs262a-F1 /ecture-02•Don’t LmpOement a functLon at tKe OoZer OeveOs of tKe s\stem unOess Lt can be compOeteO\ LmpOemented at tKLs OeveO•UnOess \ou can reOLeve tKe burden from Kosts don’t botKer cs262a-F1 /ecture-02•TKLnN tZLce before LmpOementLnJ functLonaOLt\ Ln tKe •,f Kosts can LmpOement functLonaOLt\ correctO\ LmpOement Lt Ln a OoZer Oa\er •But do so onO\ Lf Lt does not Lmpose burdenappOLcatLons tKat do not requLre tKat functLonaOLt\•TKLs Ls tKe LnterpretatLon , use cs262a-F1 /ecture-02•*oaO FOe[LbOe netZorNLnJ protocoOs Ln support of error resLOLent vLdeo codecs•TarJet domaLn /Lve vLdeo streamLnJ over 2* *SM ceOOuOar netZorN•EnvLronment /oZ-bLt rate vLdeo codecs tKat are KLJKO\ toOerant of errors Ln tKe b\te stream–+.263 MotLon vectors predLctLon error/Ooss conceaOment•:Kat Ls tKe roOe of reOLabLOLt\ Ln tKe netZorN? cs262a-F1 /ecture-02 UnL[ BSDL 3.0 UnL[ BSDL 3.0 RTP +.263 Encoder UDP / UDP/Lte PPP / PPP/Lte RTP +.263 Decoder UDP / UDP/Lte PPP / PPP/Lte RadLo /LnN ProtocoO RadLo /LnN ProtocoO cs262a-F1 /ecture-02•*SM RadLo /LnN ProtocoO reOLabOe data deOLver\ on radLo OLnN–,ssue reOLabLOLt\ versus deOa\ –do \ou need reOLabLOLt\ ZKen •SoOutLon #1 turn off reOLabLOLt\ Ln R/P and PPP–FeZer vLdeo pacNets deOLvered!–UDP cKecNsums caused ³damaJed´ pacNets to be dropped cs262a-F1 /ecture-02Need VarLabOeDeJrees of ReOLabLOLt\•Protect Keaders need for routLnJ/deOLver\ not data!•SoOutLon #2–UDP /Lte /ar]on DeJemarN and PLnN»FOe[LbOe cKecNsum onO\ prot–R/P /Lte / PPP /Lte neZ protocoOs»Same as UDP /Lte but for radLo OLnN / OLnN Oa\er•SLmuOatLon/e[perLments UDP /Lte/R/P /Lte/PPP /Lte–CoOOected 440 mLn of ZLreOess vLdeo traces ~4 mLn per vLdeo–Bad cKanneO condLtLons sLJnaO strenJtK ~2-3 B/ER ~ 1.5%–Used sLmuOatLon to repeat e[perLments•ResuOts–/ess E2E deOa\ constant MLtter KLJKer tKrouJKput OoZer pacN cs262a-F1 /ecture-02 Mean & MLn/Ma[1.260.5110.3UDP R/PUDP R/P /Lte UDP /Lte R/P /Lte End-to-End DeOa\ s cs262a-F1 /ecture-02 Mean & MLn/Ma[1.05%2.0%UDP R/PUDP R/P /Lte UDP /Lte R/P /Lte PacNet /oss % cs262a-F1 /ecture-02 :areKouse/COoud •Smart ManufacturLnJ•Smart Contracts•BLJ Data AnaO\tLcs•MacKLne /earnLnJ•ControO cs262a-F1 /ecture-02AntL End-to-End ArJuments for EdJe Compute?•Trust DomaLns *roups of Resources oZned b\ sLnJOe entLt\–RefOect tKe oZnersKLp trustZortKLness and deJree of maLntena–Carr\ unLque economLc poOLtLcaO or LncentLve structure of tKe oZner–Pa\-for-servLce federated utLOLt\ modeO•Trust for –MessaJe Transport /ocatLon ResoOutLon DataCapsuOeServLce Secure EncOave ServLce SES–,nsuOatLon from denLaO of servLce attacNs–ConversatLons routed accordLnJ to DataCapsuOeoZner’s Trust Pre *OobaO TLer-1 DomaLnTrusted ServLce ProvLder *DP PeerLnJ cs262a-F1 /ecture-02•E2E arJument encouraJes us to Neep ,P sLmpOe•,f KLJKer Oa\er can LmpOement functLonaOLt\ correctO\ LmpOement Lt Ln a OoZer Oa\er –Lt Lmproves tKe performance sLJnLfLcantO\ for appOLcatLon tKat–Lt on appOLcatLons tKat do not requLre tKat •PrLncLpOe Ls broadO\ appOLcabOe to otKer s\stems domaLns–StoraJe arcKLtecture … cs262a-F1 /ecture-02,s tKLs a Jood paper?•:Kat Zere tKe autKors’ JoaOs?•:Kat about tKe evaOuatLon/metrLcs?•DLd tKe\ convLnce \ou tKat tKLs Zas a Jood •:ere tKere an\ red-fOaJs?•:Kat mLstaNes dLd tKe\ maNe?•Does tKe s\stem/approacK meet tKe ³Test of TLme´ •+oZ ZouOd \ou revLeZ tKLs paper toda\? cs262a-F1 /ecture-02DataBaseManaJement S\stems +Lstor\•/ate 60’s netZorN CODAS & KLerarcKLcaO ,MS DBMS–,MS buLOt for ApoOOo proJram–CKarOes BacKman fatKer of CODAS predecessor ,DS at *E Ln e160’s TurLnJ aZard # 13 betZeen DLMNstraand .nutK cs262a-F1 /ecture-02NetZorN ModeO E[ampOe cs262a-F1 /ecture-02•,MS E[ampOe –One Ls parent one Ls cKLOd–ProbOems LncOude redundanc\ and requLrement of KavLnJ a parentdeOetLon anomaOLes –/oZ-OeveO ³record-at-a-tLme’´ data manLpuOatLon OanJuaJe L.e. pK\sLcaO data structures refOected Ln DM/ no data cs262a-F1 /ecture-0210 EdJar Codd’sPaper•TKe most LnfOuentLaO paper Ln DB researcK–Set-at-a-tLme DM/ZLtK tKe Ne\ Ldea of ³data Lndependence´–AOOoZs for scKema and pK\sLca–PapadLmLtrLou ³as cOear a paradLJm sKLft as Ze can Kope to fL–EdJar F. Codd TurLnJ aZard #1 11 betZeen +oare and CooN cs262a-F1 /ecture-02•:Kat pK\sLcaO trLcNs couOd \ou pOa\ under tKe covers? TKLnN about modern +:! •³+eOOersteLn’s,nequaOLt\´ –Need data Lndependence ZKen dt–OtKer scenarLos ZKere tKLs KoOds? –TKLs Ls an earO\ poZerfuO Lnstance of tZo tKemes levels of cs262a-F1 /ecture-02³Modern´ DBMS Protot\pes•MLd 0’s –:KoOesaOe adoptLon of Codd’svLsLon Ln 2 fuOO-functLon sort of–Ancestors of essentLaOO\ aOO toda\’s commercLaO s\stems •,nJres and S\stem R cs262a-F1 /ecture-02 •An earO\ and pLoneerLnJ ³pLcNup team’´ LncOudLnJ StonebraNer& :onJ•BeJat ,nJres Corp CA CA-UnLverse BrLtton-/ee S\base MS SQ/ Server :anJ’s PACE Tandem Non-Stop SQ/ cs262a-F1 /ecture-02 •15 PKDs •BeJat ,BM s SQ//DS & DB2 OracOe +P’s AOObase Tandem Non-Stop SQ/•-Lm *ra\ TurLnJ AZard #221 betZeen EnJOebartand BrooNs •/ots of BerNeOe\ foONs on tKe S\stem R team–,ncOudLnJ *ra\ 1st CS PKD @ BerNeOe\ Bruce /Lndsa\ ,rv SeOLnJer Bob practLce! –ACM SoftZare S\stems aZard #6 sKared b\ botK –Stated JoaO of botK s\stems Zas to taNe Codd’stKeor\ and turn a ZorNabOe s\stem as fast as CODAS but mucK easLer to use and–,nterestLnJO\ StonebraNerreceLved ACM S,*MOD ,nnovatLons AZar
cs262a-F1 /ecture-02•EarO\ 0’s commercLaOL]atLon of reOatLonaO s\stems –EOOLson’s OracOe beats ,BM to marNet b\ readLnJ ZKLte papers–,BM reOeases muOtLpOe RDBMSs settOes doZn to DB2–*ra\ S\stem R -err\ +eOd ,nJres and otKers MoLn Tandem –.apaOLEsZaranstarts EsVaO ZKLcK beJets +P AOObaseand CuOOLnet–ReOatLonaO TecKnoOoJ\ ,nc,nJres Corp BrLtton-/ee/S\base –CA reOeases CA-UnLverse a commercLaOL]atLon of ,nJres –,nformL[ started b\ CaO aOum RoJer SLppOno pedLJree to resear–Teradata started b\ some CaO TenetZorNLnJ tecKnoOoJ\ no pedLJree to softZare researcK tKouJKsee paraOOeO DBMS dLscussLon Oater Ln semester!
cs262a-F1 /ecture-02•MLd 0’s SQ/ becomes ³LnterJaOactLc standard´–DB2 becomes ,BM’s fOaJsKLp product–,MS ³sunseted´
cs262a-F1 /ecture-02•NetZorN & KLerarcKLcaO are OeJac\ s\stems tKouJK commonO\ Ln use! –,MS stLOO ZLdeO\ used Ln banNLnJ aLrOLne reservatLons etc. •ReOatLonaO marNet commodLtL]ed–MLcrosoft OracOe and ,BM fLJKtLnJ over buON of marNet–NCR Teradata S\base +P Nonstop and a feZ otKers v\LnJ to –OpenSourcecomLnJ of aJe LncOudLnJ M\SQ/ PostJreSQ/ ,nJres –BerNeOe\DBLs an embedded transactLonaO store tKat Ls ZLdeO\ usas ZeOO but noZ oZned b\ OracOe–;M/ and obMect-orLented features Kave pervaded tKe reOatLonaO J tKe ³purLt\´ of Codd’svLsLon
cs262a-F1 /ecture-02•BLJ compOe[ record-NeepLnJ appOLcatLons OLNe SAP and PeopOeSoft ZKLcK run •³EnterprLse appOLcatLons´ to Neep busLnesses KummLnJ–ERP EnterprLse Resource POannLnJ SAP Baan PeopOeSoft Orac–CRM Customer ReOatLonsKLp ManaJement E.pKLpKan\ SLebeO Ora–SCM SuppO\ CKaLn ManaJement TrLOoJ\ L2 OracOe ,BM etc. –+uman Resources DLrect MarNetLnJ CaOO Center SaOes Force Au+eOp DesN CataOoJ ManaJement etc. •T\pLcaOO\ cOLent-server a S\base ³LnventLon´ ZLtK a form-based AP,–Focus on resource manaJement secondar\ to focus on data manaJe•TradLtLonaOO\ a maLn Mob of a DBMS Ls to maNe tKese NLnds of apps eas\ to ZrLte
cs262a-F1 /ecture-02•RDMBS are B,* Kard to moduOarL]e pLeces of softZare–Man\ macro and mLcro scaOe s\stem desLJn decLsLons –:e’OO focus on macro desLJn Lssues toda\ mLcro Ln future Oect•DLsN manaJement cKoLces –fLOe per reOatLon –bLJ fLOe Ln fLOe s\stem –raZ devLce •Process ModeO –process per user –server –muOtL-server •+ardZare ModeO –sKared notKLnJ –sKared memor\ –sKared dLsN
cs262a-F1 /ecture-02•BasLc moduOes –parser –quer\ reZrLte –optLmL]er –quer\ e[ecutor –access metKods –buffer manaJer –OocN manaJer –OoJ/recover\ manaJer
cs262a-F1 /ecture-02•Some ³s\stems cKestnuts´ seen Ln tKLs paper–E[pect to tKroZ out tKe 1st versLon of tKe s\stem –E[pose LnternaOs vLa standard possLbOe e.J. cataOoJs as tabOes tKe /procfLOes\stem etc. –OptLmL]e tKe fast patK –,nterpretatLon vs. compLOatLon vs. LntermedLate ³opcode´ –Component faLOure as a common case to consLder –ProbOems arLsLnJ from LnteractLons betZeen repOLcated functLon
cs262a-F1 /ecture-02•FOe[LbLOLt\ of storaJe mecKanLsms –DomaLns/LnversLons vs. Keap-fLOes/Lnde[es•Use of T,D-OLsts common Ln modern DBMS so be doctrLnaLre?:Kat about Data ,ndependence?–One ansZer \ou Kave to Jet transactLons rLJKt for eacK ³acces
cs262a-F1 /ecture-02•S\stem R Zas often CPU bound tKouJK tKat’s a coarse-JraLned assertLon --reaOO\ means NOT dLsN-bound–TKLs Ls common toda\ Ln ZeOO-provLsLoned DBMSs as ZeOO.:K\?•DBMSs are not monoOLtKLc desLJns reaOO\–TKe RSS stuff does LntertZLne OocNLnJ and OoJJLnJ Lnto dLsN acLnde[LnJ and buffer manaJement.But RDS/RSS boundar\ Ls cOean a
cs262a-F1 /ecture-02•Access controO vLa vLeZs a deep appOLcatLon of data •TransactLonaO contrLbutLon of S\stem R botK conceptuaO and LmpOementatLon as Lmportant as reOatLonaO modeO and Ln fact sKouOd be decoupOed from reOatLonaO modeO.
cs262a-F1 /ecture-02•A cOassLc cross-OeveO scKeduOLnJ LnteractLon–:e ZLOO see tKLs aJaLn! •PoorO\ e[pOaLned Ln tKe paper tKree bLJ Lssues•TZo LnteractLons betZeen OS and DB scKeduOLnJ –#1 OS can preempt a database ³process´ even ZKen tKat processLs KoOdLnJ a KLJK-traffLc DB OocN –#2 DB processes sLttLnJ Ln DB OocN queues use up tKeLr OS scKeduOLnJ quanta ZKLOe ZaLtLnJ poorO\ e[pOaLned Ln te[t tKeremoved from tKe ³muOtLproJrammLnJ set´ and Jo to ³sOeep´ –and an e[pensLve OS dLspatcK Ls requLred to run tKem aJaLn
cs262a-F1 /ecture-02•/ast Lssue Ls DBMS uses a FCFS ZaLt queue for OocN –For a KLJK-traffLc DB OocN DB processes ZLOO request Lt on av–,f tKe OS preempts a DB process KoOdLnJ tKat KLJK-traffLc DB Oqueue beKLnd tKe OocN JroZs to LncOude aOmost aOO DB processes–Moreover tKe queue Ls too OonJ to be draLned Ln T tLmesteps ³stabOe´ --ever\ DB process queues bacN up before tKe queue draOessO\ ZaLtLnJ Ln OLne afte–+ence eacK DB process Ls aZaNe for onO\ one Jrant of tKe OocN subsequent T tLmestepsof usefuO ZorN after ZKLcK tKe\ queue fo•TKe resuOt Ls tKat tKe usefuO ZorN per OS ZaNLnJ perLod Ls about T tLmesteps ZKLcK Ls sKorter tKan tKe overKead of scKeduOLnJ –Kence tKe s\stem Ls tKrasKLnJ
cs262a-F1 /ecture-02•AttacNs tKe onO\ Lssue tKat can be KandOed ZLtKout LnteractLnJ ZLtK tKe OS #3 tKe FCFS DB OocN queue–Paper’s e[pOanatLon Ls confusLnJ•PoLnt Ls to aOZa\s aOOoZ an\ one of tKe DB processes currentO\ Ln tKe ³muOtLproJrammLnJ set´ to LmmedLateO\ Jet tKe OocN ZLtKout burnLnJ a quantum ZaLtLnJ on tKe OocN–+ence no quanta are Zasted on ZaLt–TecKnLcaOO\ tKLs Ls not ³faLr´ KoZever Lt Ls ³effLcLent´!•Note tKat tKe proposed poOLc\ acKLeves tKLs ZLtKout needLnJ to NnoZ ZKLcK processes are Ln tKe OS’ muOtLproJrammLnJ set
cs262a-F1 /ecture-02•TKe protot\pes tKat aOO current s\stems are based on•BasLc arcKLtecture Ls tKe same and man\ of tKe Ldeas remaLn Ln toda\’s s\stems –OptLmL]er remaLns OarJeO\ uncKanJed –RSS/RDS dLvLde remaLns Ln man\ s\stems –SQ/ cursors dupOLcates NU//s etc. »tKe pros and cons of dupOLcates. AOternatLves? »pros and cons of NU//s. AOternatLves? »JroupLnJ and aJJreJatLon –updatabOe sLnJOe-tabOe vLeZs –beJLn/end [actat user OeveO savepoLntsand restore cataOoJs a–trLJJers !! cOusterLnJ compLOed querLes B-trees –Nest-Ooop & sort-merJe MoLn aOO MoLns 2-Za\ –duaO OoJs to support OoJ faLOure
cs262a-F1 /ecture-02•SKadoZ paJLnJ •PredLcate OocNLnJ •SQ/ OanJuaJe –DupOLcate semantLcs –SubquerLesvs. MoLns –Outer MoLn •ReMected KasKLnJ
cs262a-F1 /ecture-02•UN,; paper ³TKe most Lmportant Mob of UN,; Ls to provLde a fLOe s\stem´ –UN,; and S\stem R are botK ³LnformatLon manaJement´ s\stems! –botK aOso provLde proJrammLnJ AP,s for code •BotK provLdLnJ ZKat tKe\ tKLnN are crucLaO AP,s for
cs262a-F1 /ecture-02•Bottom-Up eOeJance of s\stem vs. Top-DoZn eOeJance –maLn JoaO of UN,; provLde a smaOO proJrammers L.e. C proJrammers buLOd on top of Lt.»TKe\ are proud tKat ³No OarJe µaccess metKod’ routLnes are reqs\stem caOOs´.OS vLeZed Lts roO»No natLve OocNLnJ mecKanLsms!–maLn JoaO of S\stem R and ,nJrLnsuOated proJrammers L.e. SQ/ scrLptLnJ from tKe s\stem»*uarantee cOearO\ defLned of data and querLes.After aOO DBMS vLeZs Lts roOe as managing data for application programmers.•Affects ZKere tKe compOe[Lt\ Joes! –to tKe s\stem or tKe end-proJrammer? –questLon ZKLcK Ls better?Ln ZKat envLronments? –foOOoZ-on questLon are Lnternet s\stems more OLNe enterprLse tradLtLonaOO\ buLOt on DBMSs or scLentLfLc/end-user apps trabuLOt over OSesand fLOes?:K\?
cs262a-F1 /ecture-02•AcKLOOes
KeeO of RDBMSs a cOosed bo[ –Cannot OeveraJe tecKnoOoJ\ ZLtKout–One soOutLon maNe tKe s\stem e[tensLbOe convLnce tKe ZorOd tdoZnOoad code Lnto tKe DBMS –AnotKer soOutLon componentL]e tKe s\stem Kard? RSS Ls Kard t•AcKLOOes
KeeO of OSes Kard to decLde on tKe "rLJKt" OeveO of abstractLon –As Ze
OO read man\ UN,; abstractLons e.J. vLrtuaO memor\ messLnJ up semantLcs.On tKe can cause too mucK proJrammer burden and messes up tKe eOeJance of tKe s\stem –One soOutLon maNe tKe s\stem e[tensLbOe convLnce tKe fanc\ adoZnOoad code Lnto tKe OS –AnotKer soOutLon componentL]e tKe s\stem Kard due to protec
cs262a-F1 /ecture-02•TradLtLonaOO\ separate communLtLes despLte subsequentO\ cOear need to LnteJrate –UN,; paper ":e taNe tKe vLeZ tusers of tKe same fLOe. TKe\ are unnecessar\ because Ze are notfaced ZLtK OarJe sLnJOe-fLOe data bases maLntaLned b\ Lndepend–S\stem R "Kas LOOustrated tKe feasLbLOLt\ of compLOLnJ a ver\OeveO data subOanJuaJe SQ/ Lnto macKLne-OeveO code".
cs262a-F1 /ecture-02,s tKLs a Jood paper?•:Kat Zere tKe autKors’ JoaOs?•:Kat about tKe evaOuatLon/metrLcs?•DLd tKe\ convLnce \ou tKat tKLs Zas a Jood •:ere tKere an\ red-fOaJs?•:Kat mLstaNes dLd tKe\ maNe?•Does tKe s\stem/approacK meet tKe ³Test of TLme´ •+oZ ZouOd \ou revLeZ tKLs paper toda\?
cs262a-F1 /ecture-02•MaLn JoaO of tKLs cOass Ls to ZorN from botK of tKese dLrectLons cuOO tKe Oessons from eacK and asN KoZ to use tKese Oessons toda\ botK ZLtKLn and OUTS,DE tKe conte[t of tKese KLstorLcaOO\ separate s\stems.