This can be generalized to any dimension brPage 9br Example of 2D gradient pic of the MATLAB demo Illustration of the gradient in 2D Example of 2D gradient pic of the MATLAB demo Gradient descent works in 2D brPage 10br 10 Generalization to multiple ID: 29294
Download Pdf The PPT/PDF document "Generic descent algorithm Generalizatio..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
3 Gradient descent (illustration) guess next step Gradient descent (illustration) fx guess 8 Example of 2D gradient: pic of the MATLAB demo Illustration of the gradient in 2D Example of 2D gradient: pic of the MATLAB demo This is just a genaralization of the derivative in two dimensions.This can be generalized to any dimension. 15 Problem 2: « ping pong effect » [S. Boyd, L. Vandenberghe, Convex Convex Optimization lect. Notes, Stanford Univ. 2004 ] Problem 2: « ping pong effect » [S. Boyd, L. Vandenberghe, Convex Convex Optimization lect. Notes, Stanford Univ. 2004 ] 16 Problem 2: (other norm dependent issues) [S. Boyd, L. Vandenberghe, Convex Convex Optimization lect. Notes, Stanford Univ. 2004 ] Problem 3: stopping criterion In multiple dimensions: Or equivalently Rarely used in practice.More about this in EE227A (convex optimization, Prof. L. El Ghaoui). 17 Several methods exist to address this problem- Line search methods, in particular- Backtracking line search- Exact line search- Normalized steepest descent- Newton stepsFundamental problem of the method: local minima Local minima: pic of the MATLAB demo The iterations of the algorithmconverge to a local minimum