CMSC Spring Learning Theory Lecture Mistake Bound Model Halving Algorithm Linear Classiers Instructors Sham Kakade and Ambuj Tewari Introduction This course will be divided into parts - Description

In each part we will make different assumptions about the data generating process Online Learning No assumptions about data generating process Worst case analysis Fundamental connections to Game Theory Statistical Learning Assume data consists of in ID: 6971 Download Pdf

Similar presentations

CMSC Spring Learning Theory Lecture Mistake Bound Model Halving Algorithm Linear Classiers Instructors Sham Kakade and Ambuj Tewari Introduction This course will be divided into parts
CMSC Spring Learning Theory Lecture Mistake Bound Model Halving Algorithm Linear Classiers Instructors Sham Kakade and Ambuj Tewari Introduction This course will be divided into parts

In each part we will make different assumptions about the data generating process Online Learning No assumptions about data generating process Worst case analysis Fundamental connections to Game Theory Statistical Learning Assume data consists of in

Machine Learning Theory MariaFlorina Balcan Lecture September st Plan Discuss the Mistake Bound model
Machine Learning Theory MariaFlorina Balcan Lecture September st Plan Discuss the Mistake Bound model

The Mistake Bound model In this lecture we study the online learning protocol In this setting the following scenario is repeated inde57356nitely 1 The algorithm receives an unlabeled example 2 The algorithm predicts a classi57356cation of this examp

Online Algorithms
Online Algorithms

Lecturer: . Yishay. . Mansour. Elad. . Walach. Alex . Roitenberg. Introduction. Up until . now, our algorithms start with . input and . work with it. suppose input arrives a little at a time, need instant .

Online Algorithms
Online Algorithms

Lecturer: . Yishay. . Mansour. Elad. . Walach. Alex . Roitenberg. Introduction. Up until . now, our algorithms start with . input and . work with it. suppose input arrives a little at a time, need instant .

COMP Computational Learning Theory Spring Department of Computer Science Tufts University Instructor Roni Khardon Lecture Scribe Roni Mistake Bounded Learning and the Perceptron Algorithm These note
COMP Computational Learning Theory Spring Department of Computer Science Tufts University Instructor Roni Khardon Lecture Scribe Roni Mistake Bounded Learning and the Perceptron Algorithm These note

1 MistakeBound Learning Mistakebound learning can be described in terms of playing an in64257nite learning game as follows 1 An adversary chooses some example and shows it to the learner 2 The learner tries to predict the label of the example 3 The

On-Line Algorithms in
On-Line Algorithms in

. Machine Learning. By:. WALEED ABDULWAHAB YAHYA AL-GOBI. MUHAMMAD BURHAN HAFEZ. KIM HYEONGCHEOL. HE RUIDAN. SHANG XINDI. . Overview. Introduction: . online learning vs. offline learning. Predicting from Expert Advice.

Journal of Machine Learning Research Su bmitted Revised Published Regularization Techniques for Learning with Matrices Sham M
Journal of Machine Learning Research Su bmitted Revised Published Regularization Techniques for Learning with Matrices Sham M

Kakade SKAKADE MICROSOFT COM Microsoft Research New England One Memorial Drive Cambridge MA 02142 USA Shai ShalevShwartz SHAIS CS HUJI AC IL School of Computer Science and Engineering The Hebrew University of Jerusalem Givat Ram Jerusalem 91904 Isra

Lecture 1
Lecture 1

Describing Inverse Problems. Syllabus. Lecture 01 Describing Inverse Problems. Lecture 02 Probability and Measurement Error, Part 1. Lecture 03 Probability and Measurement Error, Part 2 . Lecture 04 The L.

Lecture 9
Lecture 9

Inexact Theories. Syllabus. Lecture 01 Describing Inverse Problems. Lecture 02 Probability and Measurement Error, Part 1. Lecture 03 Probability and Measurement Error, Part 2 . Lecture 04 The L.

Machine Learning with Discriminative Methods
Machine Learning with Discriminative Methods

Lecture 02 . – . PAC Learning and tail bounds intro. CS 790-134 Spring 2015. Alex Berg. Today’s lecture. PAC Learning. Tail bounds…. Rectangle learning. +. -. -. -. -. -. -. +. +. +. Hypothesis .

147K - views

CMSC Spring Learning Theory Lecture Mistake Bound Model Halving Algorithm Linear Classiers Instructors Sham Kakade and Ambuj Tewari Introduction This course will be divided into parts

Published bymarina-yarberry

In each part we will make different assumptions about the data generating process Online Learning No assumptions about data generating process Worst case analysis Fundamental connections to Game Theory Statistical Learning Assume data consists of in

Tags : each part
Embed :
Pdf Download Link

Download Pdf - The PPT/PDF document "CMSC Spring Learning Theory Lecture M..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.

CMSC Spring Learning Theory Lecture Mistake Bound Model Halving Algorithm Linear Classiers Instructors Sham Kakade and Ambuj Tewari Introduction This course will be divided into parts






Presentation on theme: "CMSC Spring Learning Theory Lecture Mistake Bound Model Halving Algorithm Linear Classiers Instructors Sham Kakade and Ambuj Tewari Introduction This course will be divided into parts"— Presentation transcript:

Notethatweareignoringefciencyissueshere.WehavenotsaidanythingabouttheamountofcomputationAhastodoineachroundinordertoupdateitshypothesisfromhttoht+1.Settingthisissueasideforamoment,wehavearemarkablysimplealgorithmHALVING(C)thathasamistakeboundoflg(jCj)foranyniteconceptclassC.ForanitesetHofhypotheses,denethehypothesismajority(H)asfollows,majority(H)(x):=(+1jfh2Hjh(x)=+1gjjHj=2;�1otherwise: Algorithm1HALVING(C) C1 Ch1 majority(C1)fort=1toTdoReceivextPredictht(xt)ReceiveytCt+1 ff2Ctjf(xt)=ytght+1 majority(Ct+1)endfor Theorem2.2.ForanyniteconceptclassC,wehavemistake(HALVING(C);C))lgjCj:Proof.ThekeyideaisthatifthealgorithmmakesamistakethenatleasthalfofthehypothesisinCtareeliminated.Formally,ht(xt)6=yt)jCt+1jjCtj=2:Therefore,denotingthenumberofmistakesuptotimetbyMt,Mt:=TXt=11[ht(xt)6=yt];wehavejCt+1jjC1j 2Mt=jCj 2Mt(1)Sincethereisanf2Cwhichperfectlyclassiesallxt,wealsohave1jCt+1j:(2)Combining(1)and(2),wehave1jCj 2Mt;whichgivesMtlg(jCj). 3LinearClassiersandMarginLetusnowlookataconcreteexampleofaconceptclass.SupposeX=Rdandwehaveavectorw2Rd.Wedenethehypothesis,hw(x)=sgn(wx);2 Thisboundisnicebecauseeventhoughwehadanuncountableconceptclasstobeginwith,themarginassumptionallowedustoworkwithanitesubsetoftheconceptclassandwewereabletoderiveamistakebound.However,theresultisunsatisfactorybecauserunningthehalvingalgorithmonC linisextremelyinefcient.Onemightwonderifonecanusethespecialstructureofthespaceoflinearclassierstoimplementthehalvingalgorithmmoreefciently.Indeed,itpossibletoimplementavariantofthehalvingalgorithmefcientlyusingtheellipsoidmethoddevelopedforthelinearprogrammingfeasibilityproblem.Notethatthemistakebounddependsexplicitlyonthedimensiondoftheproblem.Wewouldalsoliketobeabletogiveadimensionindependentmistakebound.Indeed,aclassicalgorithmcalledPERCEPTRONhassuchamistakebound.4