Numerical differentiation

Definition

Numerical differentiation refers to a method for computing the approximate numerical value of the derivative of a function at a point in the domain as a difference quotient. Explicitly, the numerical derivative $f^{\prime }(x)$ of a function $f$ at a point $x$ may be computed using either of these three formulas, for $h$ a sufficiently small positive real number:

Expression	Interpretation of limit as $h\to 0$
Forward difference quotient $\frac{f(x+h)-f(x)}{h}$, comes from the forward difference form of the finite difference	The right-hand derivative $f{\text{'}}_{+}(x)$. If $f$ is differentiable at $x$, this equals the two-sided derivative $f^{\prime }(x)$.
Backward difference quotient $\frac{f(x)-f(x-h)}{h}$, comes from the backward difference form of the finite difference	The left-hand derivative $f{\text{'}}_{-}(x)$. If $f$ is differentiable at $x$, this equals the two-sided derivative $f^{\prime }(x)$.
Central difference quotient $\frac{f(x+(h/2))-f(x-(h/2))}{h}$, comes from the central difference form of the finite difference	If $f$ is differentiable at $x$, this equals the two-sided derivative $f^{\prime }(x)$. Otherwise, however, it does not have any direct interpretation as a one-sided derivative of $f$.

Relative precision of the formulas

Suppose that $f$ has a Taylor series around $x$. In other words, we can expand:

$f(x+h)=f(x)+h{f}^{\prime }(x)+\frac{h^2}{2}{f}^{″}(x)+\frac{h^3}{3!}{f}^{‴}(x)+\dots$

In this case, the three methods for approximating the derivative give us the following results:

Method	Computed approximate value to $f^{\prime }(x)$	Error term (computed value minus actual value)	Order of convergence (smallest exponent on $h$ with nonzero coefficient in Taylor expansion of error) -- higher order is better
forward difference quotient $\frac{f(x+h)-f(x)}{h}$	$f^{\prime }(x)+\frac{h}{2}{f}^{″}(x)+\frac{h^3}{6}{f}^{‴}(x)+\dots$	$\frac{h}{2}{f}^{″}(x)+\frac{h^2}{6}{f}^{‴}(x)+\dots$	1
backward difference quotient $\frac{f(x)-f(x-h)}{h}$	$f^{\prime }(x)-\frac{h}{2}{f}^{″}(x)+\frac{h^3}{6}{f}^{‴}(x)+\dots$	$-\frac{h}{2}{f}^{″}(x)+\frac{h^2}{6}{f}^{‴}(x)+\dots$	1
central difference quotient $\frac{f(x+(h/2))-f(x-(h/2))}{h}$	$f^{\prime }(x)+\frac{h^2}{12}{f}^{‴}(x)+\dots$	$\frac{h^2}{12}{f}^{‴}(x)+\dots$	2

We therefore see that the central difference quotient computes a substantially more precise value for the derivative.

Note that we for the result above to hold, we do not require the function to have a Taylor series; we only need the function to be three or more times continuously differentiable (in fact, a somewhat weaker version holds if the function is only twice continuously differentiable).