Summary table of multivariable derivatives

This page is a summary table of multivariable derivatives.

• TODO maybe good to have separate rows for evaluated and pre-evaluated versions, for things that are functions/can be applied

Single-variable real function

For comparison and completeness, we give a summary table of the single-variable derivative. Let $f\colon \mathbf {R} \to \mathbf {R}$ be a single-variable real function.

Term	Notation	Type	Definition	Notes
Derivative of $f$	$f'$ or ${\frac {df}{dx}}$	$\mathbf {R} \to \mathbf {R}$	$f'(x)=\lim _{h\to 0}{\frac {f(x+h)-f(x)}{h}}$
Derivative of $f$ at $x_{0}\in \mathbf {R}$	$f'(x_{0})$ or ${\frac {df}{dx}}(x_{0})$ or $\left.{\frac {d}{dx}}f(x)\right|_{x=x_{0}}$	$\mathbf {R}$	${\begin{aligned}f'(x_{0})&=\lim _{h\to 0}{\frac {f(x_{0}+h)-f(x_{0})}{h}}\\&=\lim _{x\to x_{0}}{\frac {f(x)-f(x_{0})}{x-x_{0}}}\end{aligned}}$	In the most general multivariable case, $f'(x_{0})$ will become a linear transformation, so analogously we may wish to talk about the single-variable $f'(x_{0})$ as the function $f'(x_{0})\colon \mathbf {R} \to \mathbf {R}$ defined by $f'(x_{0})(x)=f'(x_{0})x$, where on the left side "$f'(x_{0})$" is a function and on the right side "$f'(x_{0})$" is a number. If "$f'(x_{0})$" is a function, we can evaluate it at $1$ to recover the number: $f'(x_{0})(1)$. This is pretty confusing, and in practice everyone thinks of "$f'(x_{0})$" in the single-variable case as a number, making the notation divergent; see Notational confusion of multivariable derivatives § The derivative as a linear transformation in the several variable case and a number in the single-variable case for more information.

Real-valued function of Rⁿ

Let $f\colon \mathbf {R} ^{n}\to \mathbf {R}$ be a real-valued function of $\mathbf {R} ^{n}$.

Term	Notation	Type	Definition	Notes
Partial derivative of $f$ with respect to its $j$th variable	$\partial _{j}f$ or $\partial _{x_{j}}f$ or ${\frac {\partial f}{\partial x_{j}}}$ or $f_{x_{j}}$ or $f_{j}$	$\mathbf {R} ^{n}\to \mathbf {R}$	$\partial _{j}f(x)=\lim _{t\to 0}{\frac {f(x+te_{j})-f(x)}{t}}$	Here $e_{j}=(0,\ldots ,1,\ldots ,0)$ is the $j$th vector of the standard basis, i.e. the vector with all zeroes except a one in the $j$th spot. Therefore $x+te_{j}$ can also be written $(x_{1},\ldots ,x_{j}+t,\ldots ,x_{n})$ when broken down into components.
Gradient	$\nabla f$	$\mathbf {R} ^{n}\to \mathbf {R} ^{n}$	$\nabla f(x)=(\partial _{1}f(x),\ldots ,\partial _{n}f(x))$
Gradient at $x_{0}\in \mathbf {R} ^{n}$	$\nabla f(x_{0})$	$\mathbf {R} ^{n}$ or ${\mathcal {M}}_{1,n}(\mathbf {R} )$	$(\partial _{1}f(x_{0}),\ldots ,\partial _{n}f(x_{0}))$ or the vector $c$ such that $\lim _{x\to x_{0}}{\frac {\|f(x)-f(x_{0})-c\cdot (x-x_{0})\|}{\|x-x_{0}\|}}=0$
Directional derivative in the direction of $v$	$D_{v}f$ or $\partial _{v}f$	$\mathbf {R} ^{n}\to \mathbf {R}$	$D_{v}f(x)=\lim _{t\to 0}{\frac {f(x+tv)-f(x)}{t}}$	When $v=e_{j}$, this reduces to the $j$th partial derivative.
Total derivative with respect to the $j$th variable	${\frac {df}{dx_{j}}}$	$\mathbf {R} \to \mathbf {R}$	For $i\neq j$, we treat the variable $x_{i}=g_{i}(x_{j})$ as a function of $x_{j}$, and take the single-variable derivative with respect to $x_{j}$ (more formally, $g:\mathbf {R} \to \mathbf {R} ^{n}$ is a function such that the $j$th component Failed to parse (unknown function "\id"): {\displaystyle g_j = \id} is the identity function). From the chain rule this becomes ${\frac {df}{dx_{j}}}=\nabla f(x)\cdot g'(x)={\frac {\partial f}{\partial x_{1}}}{\frac {dx_{1}}{dx_{j}}}+\cdots +{\frac {\partial f}{\partial x_{n}}}{\frac {dx_{n}}{dx_{j}}}$

I think in this case, since $f'(x_{0})(v)$ coincides with $\nabla f(x_{0})\cdot v$, people don't usually define the derivative separately. For example, Folland in Advanced Calculus defines differentiability but not the derivative! He just says that the vector that makes a function differentiable is the gradient.

TODO: answer questions like "Is the gradient the derivative?"

Vector-valued function of R

Let $f\colon \mathbf {R} \to \mathbf {R} ^{m}$ be a vector-valued function of $\mathbf {R}$. A parametric curve (or parametrized curve) is an example of this. Since the function is vector-valued, some authors use a boldface letter like $\mathbf {f}$.

Term	Notation	Type	Definition	Notes
Velocity vector at $t$	$v(t)$ or $Df(t)$	$\mathbf {R} \to \mathbf {R} ^{m}$	$(f_{1}'(t),\ldots ,f_{n}'(t))$

Note the absence for partial/directional derivatives. There is only one variable with respect to which we can differentiate, so there is no direction to choose from.

Vector-valued function of Rⁿ

Let $f\colon \mathbf {R} ^{n}\to \mathbf {R} ^{m}$ be a vector-valued function of $\mathbf {R} ^{n}$. Since the function is vector-valued, some authors use a boldface letter like $\mathbf {f}$.

Term	Notation	Type	Definition	Notes
Partial derivative with respect to the $j$th variable	$\partial _{j}f$ or $\partial _{x_{j}}f$ or ${\frac {\partial f}{\partial x_{j}}}$ or $f_{x_{j}}$ or $f_{j}$	$\mathbf {R} ^{n}\to \mathbf {R} ^{m}$	$\partial _{j}f(x)=\lim _{t\to 0}{\frac {f(x+te_{j})-f(x)}{t}}$
Directional derivative in the direction of $v$	$D_{v}f$ or $\partial _{v}f$	$\mathbf {R} ^{n}\to \mathbf {R} ^{m}$	$D_{v}f(x)=\lim _{t\to 0}{\frac {f(x+tv)-f(x)}{t}}$
Total or Fréchet derivative (sometimes just called the derivative) at point $x_{0}\in \mathbf {R} ^{n}$	$f'(x_{0})$ or $(Df)_{x_{0}}$ or $d_{x_{0}}f$	$\mathbf {R} ^{n}\to \mathbf {R} ^{m}$	The linear transformation $L$ such that $\lim _{x\to x_{0}}{\frac {\|f(x)-f(x_{0})-L(x-x_{0})\|}{\|x-x_{0}\|}}=0$	The derivative at a given point is a linear transformation. One might wonder then what the derivative (without giving a point) is, i.e. what meaning to assign to "$f'$" as we can in the single-variable case. Its type would have to be $\mathbf {R} ^{n}\to \mathbf {R} ^{n}\to \mathbf {R} ^{m}$ or more specifically $\mathbf {R} ^{n}\to {\mathcal {L}}(\mathbf {R} ^{n},\mathbf {R} ^{m})$ (where ${\mathcal {L}}(\mathbf {R} ^{n},\mathbf {R} ^{m})$ is the set of linear transformations from $\mathbf {R} ^{n}$ to $\mathbf {R} ^{m}$). Also the notation $f'(x_{0})$ is slightly confusing: if the total derivative is a function, what happens if $n=m=1$? We see that $f'(x_{0})\colon \mathbf {R} \to \mathbf {R}$, so the single-variable derivative isn't actually a number! To get the actual slope of the tangent line, we must evaluate the function at $1$: $f'(x_{0})(1)\in \mathbf {R}$. Some authors avoid this by using different notation in the general multivariable case. Others accept this type error and ignore it.
Derivative matrix, differential matrix, Jacobian matrix at point $x_{0}\in \mathbf {R} ^{n}$	$Df(x_{0})$ or ${\mathcal {M}}(f'(x_{0}))$	${\mathcal {M}}_{m,n}(\mathbf {R} )$	${\begin{pmatrix}\partial _{1}f_{1}(x_{0})&\cdots &\partial _{n}f_{1}(x_{0})\\\vdots &\ddots &\vdots \\\partial _{1}f_{n}(x_{0})&\cdots &\partial _{n}f_{n}(x_{0})\end{pmatrix}}$	Since the total derivative is a linear transformation, and since linear transformations from $\mathbf {R} ^{n}$ to $\mathbf {R} ^{m}$ have a one-to-one correspondence with real-valued $m$ by $n$ matrices, the behavior of the total derivative can be summarized in a matrix; that summary is the derivative matrix. Some authors say that the total derivative is the matrix. TODO: talk about gradient vectors as rows.

Note the absence of the gradient in the above table. The generalization of the gradient to the $\mathbf {R} ^{n}\to \mathbf {R} ^{m}$ case is the derivative matrix.

See also

• Notational confusion of multivariable derivatives
• Relation between gradient vector and partial derivatives
• Relation between gradient vector and directional derivatives
• Directional derivative
• machinelearning:Summary table of probability terms

References

• Tao, Terence. Analysis II. 2nd ed. Hindustan Book Agency. 2009.
• Folland, Gerald B. Advanced Calculus. Pearson. 2002.
• Pugh, Charles Chapman. Real Mathematical Analysis. Springer. 2010.