Gradient descent using Newton's method: Difference between revisions
(Created page with "==Definition== '''Gradient descent using Newton's method''' is a variant of gradient descent where the step size along the gradient descent is determined using Newton's...") |
No edit summary |
||
| Line 2: | Line 2: | ||
'''Gradient descent using Newton's method''' is a variant of [[gradient descent]] where the step size along the gradient descent is determined using [[Newton's method for optimization of a function of one variable|Newton's method]]. In other words, we move the same way that we would move if we were applying Newton's method to the function restricted to the line of the gradient vector through the point. | '''Gradient descent using Newton's method''' is a variant of [[gradient descent]] where the step size along the gradient descent is determined using [[Newton's method for optimization of a function of one variable|Newton's method]]. In other words, we move the same way that we would move if we were applying Newton's method to the function restricted to the line of the gradient vector through the point. | ||
By default, we are referring to gradient descent using ''one'' iteration of Newton's method, i.e., we stop Newton's method after one iteration. However, we can consider gradient descent using Newton's method where we use multiple iterations of Newton's method to determine the step size for gradient descent. | |||
Revision as of 02:30, 31 May 2014
Definition
Gradient descent using Newton's method is a variant of gradient descent where the step size along the gradient descent is determined using Newton's method. In other words, we move the same way that we would move if we were applying Newton's method to the function restricted to the line of the gradient vector through the point.
By default, we are referring to gradient descent using one iteration of Newton's method, i.e., we stop Newton's method after one iteration. However, we can consider gradient descent using Newton's method where we use multiple iterations of Newton's method to determine the step size for gradient descent.