CMSC Spring Learning Theory Lecture Mistake Bound Model Halving Algorithm Linear Classiers Instructors Sham Kakade and Ambuj Tewari Introduction This course will be divided into parts - PDF document

Download presentation
CMSC Spring Learning Theory Lecture Mistake Bound Model Halving Algorithm Linear Classiers Instructors Sham Kakade and Ambuj Tewari Introduction This course will be divided into parts
CMSC Spring Learning Theory Lecture Mistake Bound Model Halving Algorithm Linear Classiers Instructors Sham Kakade and Ambuj Tewari Introduction This course will be divided into parts

CMSC Spring Learning Theory Lecture Mistake Bound Model Halving Algorithm Linear Classiers Instructors Sham Kakade and Ambuj Tewari Introduction This course will be divided into parts - Description

In each part we will make different assumptions about the data generating process Online Learning No assumptions about data generating process Worst case analysis Fundamental connections to Game Theory Statistical Learning Assume data consists of in ID: 6971 Download Pdf

Tags

each part

Embed / Share - CMSC Spring Learning Theory Lecture Mistake Bound Model Halving Algorithm Linear Classiers Instructors Sham Kakade and Ambuj Tewari Introduction This course will be divided into parts

Presentation on theme: "CMSC Spring Learning Theory Lecture Mistake Bound Model Halving Algorithm Linear Classiers Instructors Sham Kakade and Ambuj Tewari Introduction This course will be divided into parts"— Presentation transcript

Notethatweareignoringefciencyissueshere.WehavenotsaidanythingabouttheamountofcomputationAhastodoineachroundinordertoupdateitshypothesisfromhttoht+1.Settingthisissueasideforamoment,wehavearemarkablysimplealgorithmHALVING(C)thathasamistakeboundoflg(jCj)foranyniteconceptclassC.ForanitesetHofhypotheses,denethehypothesismajority(H)asfollows,majority(H)(x):=(+1jfh2Hjh(x)=+1gjjHj=2;�1otherwise: Algorithm1HALVING(C) C1 Ch1 majority(C1)fort=1toTdoReceivextPredictht(xt)ReceiveytCt+1 ff2Ctjf(xt)=ytght+1 majority(Ct+1)endfor Theorem2.2.ForanyniteconceptclassC,wehavemistake(HALVING(C);C))lgjCj:Proof.ThekeyideaisthatifthealgorithmmakesamistakethenatleasthalfofthehypothesisinCtareeliminated.Formally,ht(xt)6=yt)jCt+1jjCtj=2:Therefore,denotingthenumberofmistakesuptotimetbyMt,Mt:=TXt=11[ht(xt)6=yt];wehavejCt+1jjC1j 2Mt=jCj 2Mt(1)Sincethereisanf2Cwhichperfectlyclassiesallxt,wealsohave1jCt+1j:(2)Combining(1)and(2),wehave1jCj 2Mt;whichgivesMtlg(jCj). 3LinearClassiersandMarginLetusnowlookataconcreteexampleofaconceptclass.SupposeX=Rdandwehaveavectorw2Rd.Wedenethehypothesis,hw(x)=sgn(wx);2 Thisboundisnicebecauseeventhoughwehadanuncountableconceptclasstobeginwith,themarginassumptionallowedustoworkwithanitesubsetoftheconceptclassandwewereabletoderiveamistakebound.However,theresultisunsatisfactorybecauserunningthehalvingalgorithmonC linisextremelyinefcient.Onemightwonderifonecanusethespecialstructureofthespaceoflinearclassierstoimplementthehalvingalgorithmmoreefciently.Indeed,itpossibletoimplementavariantofthehalvingalgorithmefcientlyusingtheellipsoidmethoddevelopedforthelinearprogrammingfeasibilityproblem.Notethatthemistakebounddependsexplicitlyonthedimensiondoftheproblem.Wewouldalsoliketobeabletogiveadimensionindependentmistakebound.Indeed,aclassicalgorithmcalledPERCEPTRONhassuchamistakebound.4

Shom More....
By: marina-yarberry
Views: 148
Type: Public

Download Section

Please download the presentation after appearing the download area.

Download Pdf - The PPT/PDF document "CMSC Spring Learning Theory Lecture M..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.

Try DocSlides online tool for compressing your PDF Files Try Now

Related Documents

CMSC Spring Learning Theory Lecture Mistake Bound Model Halving Algorithm Linear Classiers Instructors Sham Kakade and Ambuj Tewari Introduction This course will be divided into parts
CMSC Spring Learning Theory Lecture Mistake Bound Model Halving Algorithm Linear Classiers Instructors Sham Kakade and Ambuj Tewari Introduction This course will be divided into parts PDF document
CMSC Spring Learning Theory Lecture Mistake Bound Model Halving Algorithm Linear Classiers Instructors Sham Kakade and Ambuj Tewari Introduction This course will be divided into parts PDF document
pdf 208 views
Machine Learning Theory MariaFlorina Balcan Lecture September st Plan Discuss the Mistake Bound model
Machine Learning Theory MariaFlorina Balcan Lecture September st Plan Discuss the Mistake Bound model PDF document
Machine Learning Theory MariaFlorina Balcan Lecture September st Plan Discuss the Mistake Bound model PDF document
pdf 119 views
Online Algorithms
Online Algorithms PDF document
Online Algorithms PDF document
pptx 42 views
Online Algorithms
Online Algorithms PDF document
Online Algorithms PDF document
pptx 41 views
COMP Computational Learning Theory Spring Department of Computer Science Tufts University Instructor Roni Khardon Lecture Scribe Roni Mistake Bounded Learning and the Perceptron Algorithm These note
COMP Computational Learning Theory Spring Department of Computer Science Tufts University Instructor Roni Khardon Lecture Scribe Roni Mistake Bounded Learning and the Perceptron Algorithm These note PDF document
COMP Computational Learning Theory Spring Department of Computer Science Tufts University Instructor Roni Khardon Lecture Scribe Roni Mistake Bounded Learning and the Perceptron Algorithm These note PDF document
pdf 214 views
On-Line Algorithms in
On-Line Algorithms in PDF document
On-Line Algorithms in PDF document
pptx 61 views
Journal of Machine Learning Research Su bmitted Revised Published Regularization Techniques for Learning with Matrices Sham M
Journal of Machine Learning Research Su bmitted Revised Published Regularization Techniques for Learning with Matrices Sham M PDF document
Journal of Machine Learning Research Su bmitted Revised Published Regularization Techniques for Learning with Matrices Sham M PDF document
pdf 272 views
Lecture 1
Lecture 1 PDF document
Lecture 1 PDF document
pptx 21 views
Lecture 9
Lecture 9 PDF document
Lecture 9 PDF document
pptx 24 views
Machine Learning with Discriminative Methods
Machine Learning with Discriminative Methods PDF document
Machine Learning with Discriminative Methods PDF document
pptx 0 views