Gradient vector

Definition at a point

Generic definition

Suppose $f$ is a function of many variables. We can view $f$ as a function of a vector variable. The gradient vector at a particular point in the domain is a vector whose direction captures the direction (in the domain) along which changes to $f$ are concentrated, and whose magnitude is the directional derivative in that direction.

If the gradient vector of $f$ exists at a point, then we say that $f$ is differentiable at that point.

Formal epsilon-delta definition

Suppose $f$ is a function of a vector variable ${\overline {x}}$. Suppose ${\overline {c}}$ is a point in the interior of the domain of $f$, i.e., $f$ is defined in an open ball centered at ${\overline {c}}$. The gradient vector of $f$ at ${\overline {c}}$, denoted $(\nabla f)({\overline {c}})$, is a vector ${\overline {v}}$ satisfying the following:

• For every $\epsilon >0$
• there exists $\delta >0$ such that
• for every ${\overline {x}}$ satisfying $0<|{\overline {x}}-{\overline {c}}|<\delta$ (in other words, ${\overline {x}}$ is in an open ball of radius $\delta$ centered at ${\overline {c}}$, but not qual to ${\overline {c}}$)
• we have $|f({\overline {x}})-f({\overline {c}})-{\overline {v}}\cdot ({\overline {x}}-{\overline {c}})|<\epsilon |{\overline {x}}-{\overline {c}}|$

Note on why the epsilon-delta definition is necessary

Intuitively, we want to define the gradient vector analogously to the derivative of a function of one variable, i.e., as the limit of the difference quotient:

$\lim _{{\overline {x}}\to {\overline {c}}}{\frac {f({\overline {x}})-f({\overline {c}})}{{\overline {x}}-{\overline {c}}}}$

Unfortunately, the above notation does not make direct sense because it is not permissible to divide a scalar by a vector. To rectify this, we revisit what the $\epsilon -\delta$ definition of the derivative says. It turns out that that $\epsilon -\delta$ definition can more readily be generalized to functions of vector variables. The key insight is to use the dot product of vectors.

{{#widget:YouTube|id=0a9NEdMHSpI}}

Definition as a function

Generic definition

Suppose $f$ is a function of many variables. We can view $f$ as a function of a vector variable. The gradient vector of $f$ is a vector-valued function (with vector outputs in the same dimension as vector inputs) defined as follows: it sends every point to the gradient vector of the function at the point. Note that the domain of the function is precisely the subset of the domain of $f$ where the gradient vector is defined.

If the gradient vector of $f$ exists at all points of the domain of $f$, we say that $f$ is differentiable everywhere on its domain.

{{#widget:YouTube|id=cg7z5auWG30}}

Relation with directional derivatives

Statement of relation

Version type	Statement
at a point, in vector notation (multiple variables)	Suppose $f$ is a function of a vector variable ${\overline {x}}=\langle x_{1},x_{2},\dots ,x_{n}\rangle$. Suppose ${\overline {u}}$ is a unit vector and ${\overline {a}}$ is a point in the domain of $f$. Suppose that the gradient vector of $f$ at ${\overline {a}}$ exists. We denote this gradient vector by $\nabla f({\overline {a}})$. Then, we have the following relationship: $D_{\overline {u}}(f)({\overline {a}})={\overline {u}}\cdot (\nabla f({\overline {a}}))$ The right side here is the dot product of vectors.
generic point, in vector notation (multiple variables)	Suppose $f$ is a function of a vector variable ${\overline {x}}=\langle x_{1},x_{2},\dots ,x_{n}\rangle$. Suppose ${\overline {u}}$ is a unit vector. We then have: $D_{\overline {u}}(f)({\overline {x}})={\overline {u}}\cdot (\nabla f({\overline {x}}))$ The right side here is a dot product of vectors. The equality holds whenever the right side makes sense.
generic point, point-free notation (multiple variables)	Suppose $f$ is a function of a vector variable ${\overline {x}}=\langle x_{1},x_{2},\dots ,x_{n}\rangle$. Suppose ${\overline {u}}$ is a unit vector. We then have: $D_{\overline {u}}(f)={\overline {u}}\cdot (\nabla f)$ The right side here is a dot product of vector-valued functions (the constant function ${\overline {u}}$ and the gradient vector of $f$). The equality holds whenever the right side makes sense.

{{#widget:YouTube|id=TvvSB2q5L1E}} {{#widget:YouTube|id=RvponqyVtFU}}

Relation with directional derivatives and partial derivatives

Fill this in later

{{#widget:YouTube|id=yINihD_bYzA}}

Proof of relation

Fill this in later