# Gradient vector

This article describes an analogue for functions of multiple variables of the following term/fact/notion for functions of one variable: derivative

## Contents

## Definition at a point

### Generic definition

Suppose is a function of many variables. We can view as a function of a *vector* variable. The gradient vector at a particular point in the domain is a vector whose direction captures the direction (in the domain) along which changes to are concentrated, and whose magnitude is the directional derivative in that direction.

If the gradient vector of exists at a point, then we say that is **differentiable** at that point.

### Formal epsilon-delta definition

Suppose is a function of a vector variable . Suppose is a point in the interior of the domain of , i.e., is defined in an open ball centered at . The gradient vector of at , denoted , is a vector satisfying the following:

- For every
- there exists such that
- for every satisfying (in other words, is in an open ball of radius centered at , but not qual to )
- we have

### Note on why the epsilon-delta definition is necessary

Intuitively, we want to define the gradient vector analogously to the derivative of a function of one variable, i.e., as the limit of the difference quotient:

Unfortunately, the above notation does not make direct sense because it is not permissible to divide a scalar by a vector. To rectify this, we revisit what the definition of the derivative says. It turns out that that definition can more readily be generalized to functions of vector variables. The key insight is to use the dot product of vectors.

## Definition as a function

### Generic definition

Suppose is a function of many variables. We can view as a function of a *vector* variable. The gradient vector of is a vector-valued function (with vector outputs in the same dimension as vector inputs) defined as follows: it sends every point to the gradient vector of the function at the point. Note that the domain of the function is precisely the subset of the domain of where the gradient vector is defined.

If the gradient vector of exists at all points of the domain of , we say that is differentiable everywhere on its domain.

## Relation with directional derivatives and partial derivatives

### Relation with directional derivatives

`For further information, refer: Relation between gradient vector and directional derivatives`

Version type | Statement |
---|---|

at a point, in vector notation (multiple variables) | Suppose is a function of a vector variable . Suppose is a unit vector and is a point in the domain of . Suppose that the gradient vector of at exists. We denote this gradient vector by . Then, we have the following relationship: The right side here is the dot product of vectors. |

generic point, in vector notation (multiple variables) | Suppose is a function of a vector variable . Suppose is a unit vector. We then have: The right side here is a dot product of vectors. The equality holds whenever the right side makes sense. |

generic point, point-free notation (multiple variables) | Suppose is a function of a vector variable . Suppose is a unit vector. We then have: The right side here is a dot product of vector-valued functions (the constant function and the gradient vector of ). The equality holds whenever the right side makes sense. |

### Relation with partial derivatives

`For further information, refer: Relation between gradient vector and partial derivatives`

Version type | Statement |
---|---|

at a point, in multivariable notation | Suppose is a real-valued function of variables . Suppose is a point in the domain of such that the gradient vector of at , denoted , exists. Then, the partial derivatives of with respect to all variables exist, and the coordinates of the gradient vector are the partial derivatives. In other words: |

generic point, in multivariable notation | Suppose is a real-valued function of variables . Then, we have . Equality holds wherever the left side makes sense. |

generic point, point-free notation | Suppose is a function of variables . Then, we have . Equality holds wherever the left side makes sense. |

### Note on continuous partials

`For further information, refer: Continuous partials implies differentiable`

This says that if all the partial derivatives of a function are continuous at and around a point in the domain, then the function is in fact differentiable, hence the gradient vector is described in terms of the partial derivatives as described above.

In particular, if all the partials exist and are continuous everywhere, the gradient vector exists everywhere and is given as described above.

Note that this is significant because, *a priori* (i.e., without checking continuity), knowledge of the partials tells us what the gradient vector should be if it exists, but it doesn't tell us whether the gradient vector does exist. Continuity helps bridge that knowledge gap.

## Graphical interpretation

### For a function of two variables

Suppose is a function of two variables and suppose is a point in the domain. We say that is differentiable at a point if the gradient vector exists at the point. This is equivalent to the graph of the function having a well defined tangent plane at . Further, the equation of this tangent plane is given by:

Another way of putting this is:

Note that it is possible that the partial derivatives both exist but the function is not differentiable. In this case, the surface does not have a well defined tangent plane at the point. Even though we can define a plane by the equation above, this is not the tangent plane, because the tangent plane does not exist.

### For a function of multiple variables

Suppose is a function of multiple variables and suppose is a point in the domain of . We say that is differentiable at if the gradient vector exists. This is equivalent to the graph of the function having a well defined tangent hyperplane at the point . The equation of the tangent hyperplane is given by: