Gradient vector
This article describes an analogue for functions of multiple variables of the following term/fact/notion for functions of one variable: derivative
Contents
Definition at a point
Generic definition
Suppose is a function of many variables. We can view
as a function of a vector variable. The gradient vector at a particular point in the domain is a vector whose direction captures the direction (in the domain) along which changes to
are concentrated, and whose magnitude is the directional derivative in that direction.
If the gradient vector of exists at a point, then we say that
is differentiable at that point.
Formal epsilon-delta definition
Suppose is a function of a vector variable
. Suppose
is a point in the interior of the domain of
, i.e.,
is defined in an open ball centered at
. The gradient vector of
at
, denoted
, is a vector
satisfying the following:
- For every
- there exists
such that
- for every
satisfying
(in other words,
is in an open ball of radius
centered at
, but not qual to
)
- we have
Note on why the epsilon-delta definition is necessary
Intuitively, we want to define the gradient vector analogously to the derivative of a function of one variable, i.e., as the limit of the difference quotient:
Unfortunately, the above notation does not make direct sense because it is not permissible to divide a scalar by a vector. To rectify this, we revisit what the definition of the derivative says. It turns out that that
definition can more readily be generalized to functions of vector variables. The key insight is to use the dot product of vectors.
Definition as a function
Generic definition
Suppose is a function of many variables. We can view
as a function of a vector variable. The gradient vector of
is a vector-valued function (with vector outputs in the same dimension as vector inputs) defined as follows: it sends every point to the gradient vector of the function at the point. Note that the domain of the function is precisely the subset of the domain of
where the gradient vector is defined.
If the gradient vector of exists at all points of the domain of
, we say that
is differentiable everywhere on its domain.
Relation with directional derivatives and partial derivatives
Relation with directional derivatives
For further information, refer: Relation between gradient vector and directional derivatives
Version type | Statement |
---|---|
at a point, in vector notation (multiple variables) | Suppose ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() The right side here is the dot product of vectors. |
generic point, in vector notation (multiple variables) | Suppose ![]() ![]() ![]() ![]() The right side here is a dot product of vectors. The equality holds whenever the right side makes sense. |
generic point, point-free notation (multiple variables) | Suppose ![]() ![]() ![]() ![]() The right side here is a dot product of vector-valued functions (the constant function ![]() ![]() |
Relation with partial derivatives
For further information, refer: Relation between gradient vector and partial derivatives
Version type | Statement |
---|---|
at a point, in multivariable notation | Suppose ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
generic point, in multivariable notation | Suppose ![]() ![]() ![]() ![]() Equality holds wherever the left side makes sense. |
generic point, point-free notation | Suppose ![]() ![]() ![]() ![]() |
Note on continuous partials
For further information, refer: Continuous partials implies differentiable
This says that if all the partial derivatives of a function are continuous at and around a point in the domain, then the function is in fact differentiable, hence the gradient vector is described in terms of the partial derivatives as described above.
In particular, if all the partials exist and are continuous everywhere, the gradient vector exists everywhere and is given as described above.
Note that this is significant because, a priori (i.e., without checking continuity), knowledge of the partials tells us what the gradient vector should be if it exists, but it doesn't tell us whether the gradient vector does exist. Continuity helps bridge that knowledge gap.
Graphical interpretation
For a function of two variables
Suppose is a function of two variables
and suppose
is a point in the domain. We say that
is differentiable at a point
if the gradient vector exists at the point. This is equivalent to the graph of the function having a well defined tangent plane at
. Further, the equation of this tangent plane is given by:
Another way of putting this is:
Note that it is possible that the partial derivatives both exist but the function is not differentiable. In this case, the surface does not have a well defined tangent plane at the point. Even though we can define a plane by the equation above, this is not the tangent plane, because the tangent plane does not exist.
For a function of multiple variables
Suppose is a function of multiple variables
and suppose
is a point in the domain of
. We say that
is differentiable at
if the gradient vector
exists. This is equivalent to the graph of the function having a well defined tangent hyperplane at the point
. The equation of the tangent hyperplane is given by: