Accelerated gradient method

From Calculus
Jump to: navigation, search

Definition

The term accelerated gradient method is used for variants of gradient descent that involve an "acceleration" or "momentum" term.

In terms of a gradient descent step and a momentum step

A typical accelerated gradient method carries out two steps in every iteration:

In terms of a sequence of global quadratic approximations

Fill this in later