PPT-CS B553: Algorithms for Optimization and Learning

Univariate optimization x f x Key Ideas Critical points Direct methods Exhaustive search Golden section search Root finding algorithms Bisection More next time Local vs global optimization. Univariate. optimization. x. f. (x). Key Ideas. Critical points. Direct methods. Exhaustive search. Golden section search. Root finding algorithms. Bisection. [More next time]. Local vs. global optimization. Regrets and . Kidneys. Intro to Online Stochastic Optimization. Data revealed over time. Distribution . of future events is known. Under time constraints. Limits amount of . sampling/simulation. Solve these problems with two black boxes:. Gradient descent. Key Concepts. Gradient descent. Line search. Convergence rates depend on scaling. Variants: discrete analogues, coordinate descent. Random restarts. Gradient direction . is orthogonal to the level sets (contours) of f,. : Algorithms . for Optimization and Learning. Global optimization. 1. Agenda: . Global Optimization. Local search, optimization. Branch and bound search. Online . search. 3. Global . Optimization. min . Linear programming, quadratic programming, sequential quadratic programming. Key ideas. Linear programming. Simplex method. Mixed-integer linear programming. Quadratic programming. Applications. Radiosurgery. Optimization Algorithms. Welcome!. CS4234 . Overview. Optimization Algorithms. http://. www.comp.nus.edu.sg/. ~gilbert/CS4234. Instructor: . Seth Gilbert. Office: . COM2. -323. Office hours: . by appointment. Learning. Structure . Learning. Agenda. Learning probability distributions from . example data. To what extent can Bayes net structure be learned?. Constraint methods (inferring conditional independence). Gradient descent. Key Concepts. Gradient descent. Line search. Convergence rates depend on scaling. Variants: discrete analogues, coordinate descent. Random restarts. Gradient direction . is orthogonal to the level sets (contours) of f,. Applications. Lecture 5. : Sparse optimization. Zhu Han. University of Houston. Thanks Dr. . Shaohua. Qin’s efforts on slides. 1. Outline (chapter 4). Sparse optimization models. Classic solvers and omitted solvers (BSUM and ADMM). Classification of algorithms. The DIRECT algorithm. Divided rectangles. Exploration and Exploitation as bi-objective optimization. Application to High Speed Civil Transport. Global optimization issues. Bayesian . Networks. agenda. B. ayesian networks. Chain rule for . Bayes . nets. Naïve Bayes models. Independence declarations. D-separation. Probabilistic inference queries. Purposes of . bayesian. Bayesian . Networks. agenda. Probabilistic . inference . queries. Top-down . inference. Variable elimination. Probability Queries. Given: some probabilistic model over variables . X. Find: distribution over . 10 Bat Algorithms Xin-She Yang, Nature-Inspired Optimization Algorithms, Elsevier, 2014 The bat algorithm (BA) is a bio-inspired algorithm developed by Xin-She Yang in 2010. 10.1 Echolocation of Bats Learning. Parameter Learning with Hidden Variables & . Expectation . Maximization. Agenda. Learning probability distributions from . data. in the setting of known structure, . missing data. Expectation-maximization (EM) algorithm.