PPT-CS B553

Algorithms for Optimization and Learning Global optimization 1 Agenda Global Optimization Local search optimization Branch and bound search Online search 3 Global Optimization min . Univariate. optimization. x. f. (x). Key Ideas. Critical points. Direct methods. Exhaustive search. Golden section search. Root finding algorithms. Bisection. [More next time]. Local vs. global optimization. : A. lgorithms . for Optimization and Learning. Monte Carlo Methods for . Probabilistic Inference. Agenda. Monte Carlo methods. O(1/. sqrt. (N)) standard deviation. For Bayesian inference. Likelihood weighting. Gradient descent. Key Concepts. Gradient descent. Line search. Convergence rates depend on scaling. Variants: discrete analogues, coordinate descent. Random restarts. Gradient direction . is orthogonal to the level sets (contours) of f,. Linear programming, quadratic programming, sequential quadratic programming. Key ideas. Linear programming. Simplex method. Mixed-integer linear programming. Quadratic programming. Applications. Radiosurgery. ̀Ā܀ༀ؀ЀȀȀ܀ ᔀ6+;B553ᔀ7.;86/.+ᔀ=/;ᔀC+5/+ᘀ+B,/;;Bᘀ5//.3712/+;=ᘀ5ᘀ/,/;;Bᜀ+6/55 : A. lgorithms . for Optimization and Learning. Monte Carlo Methods for . Probabilistic Inference. Agenda. Monte Carlo methods. O(1/. sqrt. (N)) standard deviation. For Bayesian inference. Likelihood weighting. Learning. Structure . Learning. Agenda. Learning probability distributions from . example data. To what extent can Bayes net structure be learned?. Constraint methods (inferring conditional independence). Univariate. optimization. x. f. (x). Key Ideas. Critical points. Direct methods. Exhaustive search. Golden section search. Root finding algorithms. Bisection. [More next time]. Local vs. global optimization. Gradient descent. Key Concepts. Gradient descent. Line search. Convergence rates depend on scaling. Variants: discrete analogues, coordinate descent. Random restarts. Gradient direction . is orthogonal to the level sets (contours) of f,. Bayesian . Networks. agenda. B. ayesian networks. Chain rule for . Bayes . nets. Naïve Bayes models. Independence declarations. D-separation. Probabilistic inference queries. Purposes of . bayesian. Bayesian . Networks. agenda. Probabilistic . inference . queries. Top-down . inference. Variable elimination. Probability Queries. Given: some probabilistic model over variables . X. Find: distribution over . Learning. Parameter Learning with Hidden Variables & . Expectation . Maximization. Agenda. Learning probability distributions from . data. in the setting of known structure, . missing data. Expectation-maximization (EM) algorithm.