Summary table of multivariable derivatives

This page is a summary table of multivariable derivatives.

• TODO maybe good to have separate rows for evaluated and pre-evaluated versions, for things that are functions/can be applied

Single-variable real function

For comparison and completeness, we give a summary table of the single-variable derivative. Let $f:\mathbf{R}\to \mathbf{R}$ be a single-variable real function.

Term	Notation	Type	Definition	Notes
Derivative of $f$	$f^{\prime }$ or $\frac{df}{dx}$	$\mathbf{R}\to \mathbf{R}$	$f^{\prime }(x)=\underset{h\to 0}{lim}\frac{f(x+h)-f(x)}{h}$
Derivative of $f$ at $x_0\in \mathbf{R}$	$f^{\prime }(x_0)$ or $\frac{df}{dx}(x_0)$ or ${\frac{d}{dx}f(x)|}_{x=x_0}$	$\mathbf{R}$	$\begin{array}{rr}f\text{'}(x_0)& =\underset{h\to 0}{lim}\frac{f(x_0+h)-f(x_0)}{h}\\ & =\underset{x\to x_0}{lim}\frac{f(x)-f(x_0)}{x-x_0}\end{array}$	In the most general multivariable case, $f^{\prime }(x_0)$ will become a linear transformation, so analogously we may wish to talk about the single-variable $f^{\prime }(x_0)$ as the function $f^{\prime }(x_0):\mathbf{R}\to \mathbf{R}$ defined by $f^{\prime }(x_0)(x)=f^{\prime }(x_0)x$, where on the left side "$f^{\prime }(x_0)$" is a function and on the right side "$f^{\prime }(x_0)$" is a number. If "$f^{\prime }(x_0)$" is a function, we can evaluate it at $1$ to recover the number: $f^{\prime }(x_0)(1)$. This is pretty confusing, and in practice everyone thinks of "$f^{\prime }(x_0)$" in the single-variable case as a number, making the notation divergent; see Notational confusion of multivariable derivatives § The derivative as a linear transformation in the several variable case and a number in the single-variable case for more information.

Real-valued function of Rⁿ

Let $f:{\mathbf{R}}^n\to \mathbf{R}$ be a real-valued function of ${\mathbf{R}}^n$.

Term	Notation	Type	Definition	Notes
Partial derivative of $f$ with respect to its $j$th variable	${\partial }_{j}f$ or ${\partial }_{x_j}f$ or $\frac{\partial f}{\partial x_j}$ or $f_{x_j}$ or $f_j$	${\mathbf{R}}^n\to \mathbf{R}$	${\partial }_{j}f(x)=\underset{t\to 0}{lim}\frac{f(x+t{e}_j)-f(x)}{t}$	Here $e_j=(0,\dots ,1,\dots ,0)$ is the $j$th vector of the standard basis, i.e. the vector with all zeroes except a one in the $j$th spot. Therefore $x+t{e}_j$ can also be written $(x_1,\dots ,x_j+t,\dots ,x_n)$ when broken down into components.
Gradient	$\mathrm{\nabla }f$	${\mathbf{R}}^n\to {\mathbf{R}}^n$	$\mathrm{\nabla }f(x)=({\partial }_{1}f(x),\dots ,{\partial }_{n}f(x))$
Gradient at $x_0\in {\mathbf{R}}^n$	$\mathrm{\nabla }f(x_0)$	${\mathbf{R}}^n$ or ${\mathcal{M}}_{1,n}(\mathbf{R})$	$({\partial }_{1}f(x_0),\dots ,{\partial }_{n}f(x_0))$ or the vector $c$ such that $\underset{x\to x_0}{lim}\frac{|f(x)-f(x_0)-c\cdot (x-x_0)|}{|x-x_0|}=0$
Directional derivative in the direction of $v$	$D_{v}f$ or ${\partial }_{v}f$	${\mathbf{R}}^n\to \mathbf{R}$	$D_{v}f(x)=\underset{t\to 0}{lim}\frac{f(x+tv)-f(x)}{t}$	When $v=e_j$, this reduces to the $j$th partial derivative.
Total derivative with respect to the $j$th variable	$\frac{df}{d{x}_j}$	$\mathbf{R}\to \mathbf{R}$	For $i\ne j$, we treat the variable $x_i=g_i(x_j)$ as a function of $x_j$, and take the single-variable derivative with respect to $x_j$ (more formally, $g:\mathbf{R}\to {\mathbf{R}}^n$ is a function such that the $j$th component $g_j=\mathrm{i}\mathrm{d}$ is the identity function). From the chain rule this becomes $\frac{df}{d{x}_j}=\mathrm{\nabla }f(x)\cdot g^{\prime }(x)=\frac{\partial f}{\partial x_1}\frac{d{x}_1}{d{x}_j}+\cdots +\frac{\partial f}{\partial x_n}\frac{d{x}_n}{d{x}_j}$

I think in this case, since $f^{\prime }(x_0)(v)$ coincides with $\mathrm{\nabla }f(x_0)\cdot v$, people don't usually define the derivative separately. For example, Folland in Advanced Calculus defines differentiability but not the derivative! He just says that the vector that makes a function differentiable is the gradient.

"Total derivative" is used for two different things (which coincide in the special case where they both make sense); see my answer https://math.stackexchange.com/a/3698838/35525 for details.

TODO: answer questions like "Is the gradient the derivative?"

Vector-valued function of R

Let $f:\mathbf{R}\to {\mathbf{R}}^m$ be a vector-valued function of $\mathbf{R}$. A parametric curve (or parametrized curve) is an example of this. Since the function is vector-valued, some authors use a boldface letter like $\mathbf{f}$.

Term	Notation	Type	Definition	Notes
Velocity vector at $t$	$v(t)$ or $Df(t)$	$\mathbf{R}\to {\mathbf{R}}^m$	$({f_{1^{\prime }}}^{\prime }(t),\dots ,{f_{n^{\prime }}}^{\prime }(t))$

Note the absence for partial/directional derivatives. There is only one variable with respect to which we can differentiate, so there is no direction to choose from.

Vector-valued function of Rⁿ

Let $f:{\mathbf{R}}^n\to {\mathbf{R}}^m$ be a vector-valued function of ${\mathbf{R}}^n$. Since the function is vector-valued, some authors use a boldface letter like $\mathbf{f}$.

Term	Notation	Type	Definition	Notes
Partial derivative with respect to the $j$th variable	${\partial }_{j}f$ or ${\partial }_{x_j}f$ or $\frac{\partial f}{\partial x_j}$ or $f_{x_j}$ or $f_j$	${\mathbf{R}}^n\to {\mathbf{R}}^m$	${\partial }_{j}f(x)=\underset{t\to 0}{lim}\frac{f(x+t{e}_j)-f(x)}{t}$
Directional derivative in the direction of $v$	$D_{v}f$ or ${\partial }_{v}f$	${\mathbf{R}}^n\to {\mathbf{R}}^m$	$D_{v}f(x)=\underset{t\to 0}{lim}\frac{f(x+tv)-f(x)}{t}$
Total or Fréchet derivative (sometimes just called the derivative) at point $x_0\in {\mathbf{R}}^n$	$f^{\prime }(x_0)$ or $(Df{)}_{x_0}$ or $d_{x_0}f$	${\mathbf{R}}^n\to {\mathbf{R}}^m$	The linear transformation $L$ such that $\underset{x\to x_0}{lim}\frac{|f(x)-f(x_0)-L(x-x_0)|}{|x-x_0|}=0$	The derivative at a given point is a linear transformation. One might wonder then what the derivative (without giving a point) is, i.e. what meaning to assign to "$f^{\prime }$" as we can in the single-variable case. Its type would have to be ${\mathbf{R}}^n\to {\mathbf{R}}^n\to {\mathbf{R}}^m$ or more specifically ${\mathbf{R}}^n\to \mathcal{L}({\mathbf{R}}^n,{\mathbf{R}}^m)$ (where $\mathcal{L}({\mathbf{R}}^n,{\mathbf{R}}^m)$ is the set of linear transformations from ${\mathbf{R}}^n$ to ${\mathbf{R}}^m$). Also the notation $f^{\prime }(x_0)$ is slightly confusing: if the total derivative is a function, what happens if $n=m=1$? We see that $f^{\prime }(x_0):\mathbf{R}\to \mathbf{R}$, so the single-variable derivative isn't actually a number! To get the actual slope of the tangent line, we must evaluate the function at $1$: $f^{\prime }(x_0)(1)\in \mathbf{R}$. Some authors avoid this by using different notation in the general multivariable case. Others accept this type error and ignore it.
Derivative matrix, differential matrix, Jacobian matrix at point $x_0\in {\mathbf{R}}^n$	$Df(x_0)$ or $\mathcal{M}(f^{\prime }(x_0))$	${\mathcal{M}}_{m,n}(\mathbf{R})$	$\left(\begin{array}{ccc}{\partial }_1{f}_1(x_0)& \cdots & {\partial }_n{f}_1(x_0)\\ ⋮& \ddots & ⋮\\ {\partial }_1{f}_n(x_0)& \cdots & {\partial }_n{f}_n(x_0)\end{array}\right)$	Since the total derivative is a linear transformation, and since linear transformations from ${\mathbf{R}}^n$ to ${\mathbf{R}}^m$ have a one-to-one correspondence with real-valued $m$ by $n$ matrices, the behavior of the total derivative can be summarized in a matrix; that summary is the derivative matrix. Some authors say that the total derivative is the matrix. TODO: talk about gradient vectors as rows.
Total derivative with respect to the $j$th variable	$\frac{df}{d{x}_j}$

Note the absence of the gradient in the above table. The generalization of the gradient to the ${\mathbf{R}}^n\to {\mathbf{R}}^m$ case is the derivative matrix.

See also

• Notational confusion of multivariable derivatives
• Relation between gradient vector and partial derivatives
• Relation between gradient vector and directional derivatives
• Directional derivative
• machinelearning:Summary table of probability terms

References

• Tao, Terence. Analysis II. 2nd ed. Hindustan Book Agency. 2009.
• Folland, Gerald B. Advanced Calculus. Pearson. 2002.
• Pugh, Charles Chapman. Real Mathematical Analysis. Springer. 2010.

External links

• this post does a similar thing: https://reallyeli.com/posts/total_derivative.html