PDF-Bagging and Boosting: Resamplingfor Classifier Design

SargurSrihari sriharicedarbuffaloedu Bagging 149Arcing150adaptive reweighting and combining150refers to reusing or selecting data to improve classification 149Includes both bagging and. Ata . Kaban. Motivation & beginnings. Suppose we have a learning algorithm that is guaranteed with high probability to be slightly better than random guessing – we call this a . weak learner. E.g. if an email contains the work “money” then classify it as spam, otherwise as non-spam. Reading. Ch. 18.6-18.12, 20.1-20.3.2. (Not Ch. 18.5). Outline. Different types of learning problems. Different types of learning algorithms. Supervised learning. Decision trees. Naïve Bayes. Perceptrons. CS311, Spring 2013. Linear Classifiers/SVMs. Admin. Midterm exam posted. Assignment 4 due Friday by 6pm. No office hours tomorrow. Math. Machine learning often involves a lot of math. some aspects of AI also involve some familiarity. CSE 5095: Special Topics Course. Boosting. Nhan Nguyen. Computer Science and Engineering Dept.. Boosting. Method for converting rules of thumb into a prediction rule.. Rule . of thumb. ?. Method?. Binary Classification. By . Yoav. Freund . and Robert E. . Schapire. Presented by David Leach. Original . Slides by Glenn . Rachlin. 1. Outline:. Background. On-line allocation of resources . Introduction . The Problem. The Hedge Algorithm . A “paradigmatic” method for real-time object detection . Training is slow, but detection is very fast. Key ideas. Integral images. for fast feature evaluation. Boosting. for feature selection. Attentional. Winter 2012. Daniel Weld. Slides adapted from Tom . Dietterich. , Luke Zettlemoyer, Carlos . Guestrin. , . Nick Kushmerick, Padraig Cunningham. © Daniel S. Weld. 2. Ensembles of Classifiers . Traditional approach: Use one classifier. CMPUT 615. Boosting Idea. . We have a weak classifier, i.e., it’s error rate is a little bit better than 0.5.. . . Boosting combines a lot of such weak learners to make a strong classifier (the error rate of which is much less than 0.5). Admin. Final project. Ensemble learning. Basic idea: . if one classifier works well, why not use multiple classifiers!. Ensemble learning. Basic idea: . if one classifier works well, why not use multiple classifiers!. Chong Ho (Alex) Yu. Problems of bias and variance. The bias is . the . error which results from missing a target. . For . example, if an estimated mean is 3, but the actual population value is 3.5, then the bias value is 0.5. . Data Analytics – . ITWS-4600/ITWS-6600. Group 3 Module. 11, . April . 27. , 2017. Weak Models: Bagging, Boosting, . Bootstrap Aggregation. Bootstrap aggregation (bagging). Improve . the stability and accuracy of machine learning algorithms used in statistical . Boosting. Nhan Nguyen. Computer Science and Engineering Dept.. Boosting. Method for converting rules of thumb into a prediction rule.. Rule . of thumb. ?. Method?. Binary Classification. X: set of all possible instances or examples. . Florina. . Balcan. 03/18/2015. Perceptron, Margins, Kernels. Recap from last time: Boosting. Works by creating . a series . of challenge datasets . s.t.. . even modest performance on these can . be . (Courtesy . Boris . Babenko. ). slides adapted from Svetlana . Lazebnik. Face detection and recognition. Detection. Recognition. “Sally”. Consumer application: Apple . iPhoto. http://www.apple.com/ilife/iphoto/.