# Summary table of multivariable derivatives

• TODO maybe good to have separate rows for evaluated and pre-evaluated versions, for things that are functions/can be applied

## Single-variable real function

For comparison and completeness, we give a summary table of the single-variable derivative. Let $f\colon \mathbf R \to \mathbf R$ be a single-variable real function.

Term Notation Type Definition Notes
Derivative of $f$ $f'$ or $\frac{df}{dx}$ $\mathbf R \to \mathbf R$ $f'(x) = \lim_{h\to0} \frac{f(x+h) - f(x)}{h}$
Derivative of $f$ at $x_0 \in \mathbf R$ $f'(x_0)$ or $\frac{df}{dx}(x_0)$ or $\left.\frac{d}{dx}f(x)\right|_{x=x_0}$ $\mathbf R$ \begin{align}f'(x_0) &= \lim_{h\to0} \frac{f(x_0+h) - f(x_0)}{h} \\ &= \lim_{x\to x_0} \frac{f(x) - f(x_0)}{x-x_0}\end{align} In the most general multivariable case, $f'(x_0)$ will become a linear transformation, so analogously we may wish to talk about the single-variable $f'(x_0)$ as the function $f'(x_0)\colon \mathbf R \to \mathbf R$ defined by $f'(x_0)(x) = f'(x_0)x$, where on the left side "$f'(x_0)$" is a function and on the right side "$f'(x_0)$" is a number. If "$f'(x_0)$" is a function, we can evaluate it at $1$ to recover the number: $f'(x_0)(1)$. This is pretty confusing, and in practice everyone thinks of "$f'(x_0)$" in the single-variable case as a number, making the notation divergent; see Notational confusion of multivariable derivatives § The derivative as a linear transformation in the several variable case and a number in the single-variable case for more information.

## Real-valued function of Rn

Let $f\colon \mathbf R^n \to \mathbf R$ be a real-valued function of $\mathbf R^n$.

Term Notation Type Definition Notes
Partial derivative of $f$ with respect to its $j$th variable $\partial_j f$ or $\partial_{x_j} f$ or $\frac{\partial f}{\partial x_j}$ or $f_{x_j}$ or $f_j$ $\mathbf R^n \to \mathbf R$ $\partial_j f(x) = \lim_{t \to 0} \frac{f(x + te_j) - f(x)}{t}$ Here $e_j = (0,\ldots,1,\ldots,0)$ is the $j$th vector of the standard basis, i.e. the vector with all zeroes except a one in the $j$th spot. Therefore $x + te_j$ can also be written $(x_1,\ldots, x_j + t, \ldots, x_n)$ when broken down into components.
Gradient $\nabla f$ $\mathbf R^n \to \mathbf R^n$ $\nabla f(x) = (\partial_1 f(x), \ldots, \partial_n f(x))$
Gradient at $x_0 \in \mathbf R^n$ $\nabla f(x_0)$ $\mathbf R^n$ or $\mathcal M_{1,n}(\mathbf R)$ $(\partial_1 f(x_0), \ldots, \partial_n f(x_0))$ or the vector $c$ such that $\lim_{x\to x_0} \frac{\|f(x) - f(x_0) - c\cdot (x-x_0)\|}{\|x-x_0\|} = 0$
Directional derivative in the direction of $v$ $D_v f$ or $\partial_v f$ $\mathbf R^n \to \mathbf R$ $D_v f(x) = \lim_{t \to 0} \frac{f(x + tv) - f(x)}{t}$ When $v = e_j$, this reduces to the $j$th partial derivative.

I think in this case, since $f'(x_0)(v)$ coincides with $\nabla f(x_0)\cdot v$, people don't usually define the derivative separately. For example, Folland in Advanced Calculus defines differentiability but not the derivative! He just says that the vector that makes a function differentiable is the gradient.

## Vector-valued function of R

Let $f\colon \mathbf R \to \mathbf R^m$ be a vector-valued function of $\mathbf R$. A parametric curve (or parametrized curve) is an example of this. Since the function is vector-valued, some authors use a boldface letter like $\mathbf f$.

Term Notation Type Definition Notes
Velocity vector at $t$ $v(t)$ or $Df(t)$ $\mathbf R \to \mathbf R^m$ $(f_1'(t), \ldots, f_n'(t))$

Note the absence for partial/directional derivatives. There is only one variable with respect to which we can differentiate, so there is no direction to choose from.

## Vector-valued function of Rn

Let $f\colon \mathbf R^n \to \mathbf R^m$ be a vector-valued function of $\mathbf R^n$. Since the function is vector-valued, some authors use a boldface letter like $\mathbf f$.

Term Notation Type Definition Notes
Partial derivative with respect to the $j$th variable $\partial_j f$ or $\partial_{x_j} f$ or $\frac{\partial f}{\partial x_j}$ or $f_{x_j}$ or $f_j$ $\mathbf R^n \to \mathbf R^m$ $\partial_j f(x) = \lim_{t \to 0} \frac{f(x + te_j) - f(x)}{t}$
Directional derivative in the direction of $v$ $D_v f$ or $\partial_v f$ $\mathbf R^n \to \mathbf R^m$ $D_v f(x) = \lim_{t \to 0} \frac{f(x + tv) - f(x)}{t}$
Total or Fréchet derivative (sometimes just called the derivative) at point $x_0\in \mathbf R^n$ $f'(x_0)$ or $(Df)_{x_0}$ or $d_{x_0}f$ $\mathbf R^n \to \mathbf R^m$ The linear transformation $L$ such that $\lim_{x\to x_0} \frac{\|f(x) - f(x_0) - L(x-x_0)\|}{\|x-x_0\|} = 0$ The derivative at a given point is a linear transformation. One might wonder then what the derivative (without giving a point) is, i.e. what meaning to assign to "$f'$" as we can in the single-variable case. Its type would have to be $\mathbf R^n \to \mathbf R^n \to \mathbf R^m$ or more specifically $\mathbf R^n \to \mathcal L(\mathbf R^n, \mathbf R^m)$ (where $\mathcal L(\mathbf R^n, \mathbf R^m)$ is the set of linear transformations from $\mathbf R^n$ to $\mathbf R^m$). Also the notation $f'(x_0)$ is slightly confusing: if the total derivative is a function, what happens if $n=m=1$? We see that $f'(x_0)\colon \mathbf R \to \mathbf R$, so the single-variable derivative isn't actually a number! To get the actual slope of the tangent line, we must evaluate the function at $1$: $f'(x_0)(1) \in \mathbf R$. Some authors avoid this by using different notation in the general multivariable case. Others accept this type error and ignore it.
Derivative matrix, differential matrix, Jacobian matrix at point $x_0\in \mathbf R^n$ $Df(x_0)$ or $\mathcal M(f'(x_0))$ $\mathcal M_{m,n}(\mathbf R)$ $\begin{pmatrix}\partial_1 f_1(x_0) & \cdots & \partial_n f_1(x_0) \\ \vdots & \ddots & \vdots \\ \partial_1 f_n(x_0) & \cdots & \partial_n f_n(x_0)\end{pmatrix}$ Since the total derivative is a linear transformation, and since linear transformations from $\mathbf R^n$ to $\mathbf R^m$ have a one-to-one correspondence with real-valued $m$ by $n$ matrices, the behavior of the total derivative can be summarized in a matrix; that summary is the derivative matrix. Some authors say that the total derivative is the matrix. TODO: talk about gradient vectors as rows.

Note the absence of the gradient in the above table. The generalization of the gradient to the $\mathbf R^n \to \mathbf R^m$ case is the derivative matrix.