Gradient vector

This article describes an analogue for functions of multiple variables of the following term/fact/notion for functions of one variable: derivative

Definition at a point

Generic definition

Suppose $f$ is a function of many variables. We can view $f$ as a function of a vector variable. The gradient vector at a particular point in the domain is a vector whose direction captures the direction (in the domain) along which changes to $f$ are concentrated, and whose magnitude is the directional derivative in that direction.

If the gradient vector of $f$ exists at a point, then we say that $f$ is differentiable at that point.

Formal epsilon-delta definition

Suppose $f$ is a function of a vector variable $\overline{x}$. Suppose $\overline{c}$ is a point in the interior of the domain of $f$, i.e., $f$ is defined in an open ball centered at $\overline{c}$. The gradient vector of $f$ at $\overline{c}$, denoted $(\mathrm{\nabla }f)(\overline{c})$, is a vector $\overline{v}$ satisfying the following:

• For every $ϵ>0$
• there exists $\delta >0$ such that
• for every $\overline{x}$ satisfying $0<|\overline{x}-\overline{c}|<\delta$ (in other words, $\overline{x}$ is in an open ball of radius $\delta$ centered at $\overline{c}$, but not qual to $\overline{c}$)
• we have $|f(\overline{x})-f(\overline{c})-\overline{v}\cdot (\overline{x}-\overline{c})|<ϵ|\overline{x}-\overline{c}|$

Note on why the epsilon-delta definition is necessary

Intuitively, we want to define the gradient vector analogously to the derivative of a function of one variable, i.e., as the limit of the difference quotient:

$\underset{\overline{x}\to \overline{c}}{lim}\frac{f(\overline{x})-f(\overline{c})}{\overline{x}-\overline{c}}$

Unfortunately, the above notation does not make direct sense because it is not permissible to divide a scalar by a vector. To rectify this, we revisit what the $ϵ-\delta$ definition of the derivative says. It turns out that that $ϵ-\delta$ definition can more readily be generalized to functions of vector variables. The key insight is to use the dot product of vectors.

{{#widget:YouTube|id=0a9NEdMHSpI}}

Definition as a function

Generic definition

Suppose $f$ is a function of many variables. We can view $f$ as a function of a vector variable. The gradient vector of $f$ is a vector-valued function (with vector outputs in the same dimension as vector inputs) defined as follows: it sends every point to the gradient vector of the function at the point. Note that the domain of the function is precisely the subset of the domain of $f$ where the gradient vector is defined.

If the gradient vector of $f$ exists at all points of the domain of $f$, we say that $f$ is differentiable everywhere on its domain.

{{#widget:YouTube|id=cg7z5auWG30}}

Relation with directional derivatives

Statement of relation

For further information, refer: Relation between gradient vector and directional derivatives

Version type	Statement
at a point, in vector notation (multiple variables)	Suppose $f$ is a function of a vector variable $\overline{x}=⟨x_1,x_2,\dots ,x_n⟩$. Suppose $\overline{u}$ is a unit vector and $\overline{a}$ is a point in the domain of $f$. Suppose that the gradient vector of $f$ at $\overline{a}$ exists. We denote this gradient vector by $\mathrm{\nabla }f(\overline{a})$. Then, we have the following relationship: $D_{\overline{u}}(f)(\overline{a})=\overline{u}\cdot (\mathrm{\nabla }f(\overline{a}))$ The right side here is the dot product of vectors.
generic point, in vector notation (multiple variables)	Suppose $f$ is a function of a vector variable $\overline{x}=⟨x_1,x_2,\dots ,x_n⟩$. Suppose $\overline{u}$ is a unit vector. We then have: $D_{\overline{u}}(f)(\overline{x})=\overline{u}\cdot (\mathrm{\nabla }f(\overline{x}))$ The right side here is a dot product of vectors. The equality holds whenever the right side makes sense.
generic point, point-free notation (multiple variables)	Suppose $f$ is a function of a vector variable $\overline{x}=⟨x_1,x_2,\dots ,x_n⟩$. Suppose $\overline{u}$ is a unit vector. We then have: $D_{\overline{u}}(f)=\overline{u}\cdot (\mathrm{\nabla }f)$ The right side here is a dot product of vector-valued functions (the constant function $\overline{u}$ and the gradient vector of $f$). The equality holds whenever the right side makes sense.

{{#widget:YouTube|id=TvvSB2q5L1E}} {{#widget:YouTube|id=RvponqyVtFU}}

Relation with directional derivatives and partial derivatives

If the gradient vector exists, then its coordinates are given by the partial derivatives.

{{#widget:YouTube|id=yINihD_bYzA}}

Proof of relation

Fill this in later