Numerical differentiation: Difference between revisions

Revision as of 02:40, 31 May 2014

Definition

Numerical differentiation refers to a method for computing the approximate numerical value of the derivative of a function at a point in the domain as a difference quotient. Explicitly, the numerical derivative $f^{'} (x)$ of a function $f$ at a point $x$ may be computed using either of these three formulas, for $h$ a sufficiently small positive real number:

Expression	Interpretation of limit as $h \to 0$
Forward difference quotient $\frac{f (x + h) - f (x)}{h}$ , comes from the forward difference form of the finite difference	The right-hand derivative $f'_{+} (x)$ . If $f$ is differentiable at $x$ , this equals the two-sided derivative $f^{'} (x)$ .
Backward difference quotient $\frac{f (x) - f (x - h)}{h}$ , comes from the backward difference form of the finite difference	The left-hand derivative $f'_{-} (x)$ . If $f$ is differentiable at $x$ , this equals the two-sided derivative $f^{'} (x)$ .
Central difference quotient $\frac{f (x + (h / 2)) - f (x - (h / 2))}{h}$ , comes from the central difference form of the finite difference	If $f$ is differentiable at $x$ , this equals the two-sided derivative $f^{'} (x)$ . Otherwise, however, it does not have any direct interpretation as a one-sided derivative of $f$ .

Relative precision of the formulas

Suppose that $f$ has a Taylor series around $x$ . In other words, we can expand:

$f (x + h) = f (x) + h f^{'} (x) + \frac{h^{2}}{2} f^{″} (x) + \frac{h^{3}}{3!} f^{‴} (x) + \dots$

In this case, the three methods for approximating the derivative give us the following results:

Method	Computed approximate value to $f^{'} (x)$	Error term (computed value minus actual value)	Order of convergence (smallest exponent on $h$ with nonzero coefficient in Taylor expansion of error) -- higher order is better
forward difference quotient $\frac{f (x + h) - f (x)}{h}$	$f^{'} (x) + \frac{h}{2} f^{″} (x) + \frac{h^{2}}{6} f^{‴} (x) + \dots$	$\frac{h}{2} f^{″} (x) + \frac{h^{2}}{6} f^{‴} (x) + \dots$	1
backward difference quotient $\frac{f (x) - f (x - h)}{h}$	$f^{'} (x) - \frac{h}{2} f^{″} (x) + \frac{h^{2}}{6} f^{‴} (x) + \dots$	$- \frac{h}{2} f^{″} (x) + \frac{h^{3}}{6} f^{‴} (x) + \dots$	1
central difference quotient $\frac{f (x + (h / 2)) - f (x - (h / 2))}{h}$	$f^{'} (x) + \frac{h^{2}}{12} f^{‴} (x) + \dots$	$\frac{h^{2}}{12} f^{‴} (x) + \dots$	2

We therefore see that the central difference quotient computes a substantially more precise value for the derivative.

Note that we for the result above to hold, we do not require the function to have a Taylor series; we only need the function to be three or more times continuously differentiable (in fact, a somewhat weaker version holds if the function is only twice continuously differentiable).

@@ Line 10: / Line 10: @@
 | Backward [[difference quotient]] <math>\frac{f(x) - f(x - h)}{h}</math>, comes from the backward difference form of the [[finite difference]] || The left-hand derivative <math>f'_-(x)</math>. If <math>f</math> is differentiable at <math>x</math>, this equals the two-sided derivative <math>f'(x)</math>.
 |-
-| Central difference quotient <math>\frac{f(x + h) - f(x - h)}{2h}</math>, comes from the central difference form of the [[finite difference]] || If <math>f</math> is differentiable at <math>x</math>, this equals the two-sided derivative <math>f'(x)</math>. Otherwise, however, it does not have any direct interpretation as a one-sided derivative of <math>f</math>.
+| Central difference quotient <math>\frac{f(x + (h/2)) - f(x - (h/2))}{h}</math>, comes from the central difference form of the [[finite difference]] || If <math>f</math> is differentiable at <math>x</math>, this equals the two-sided derivative <math>f'(x)</math>. Otherwise, however, it does not have any direct interpretation as a one-sided derivative of <math>f</math>.
 |}
+==Relative precision of the formulas==
+Suppose that <math>f</math> has a Taylor series around <math>x</math>. In other words, we can expand:
+<math>f(x + h) = f(x) + hf'(x) + \frac{h^2}{2}f''(x) + \frac{h^3}{3!}f'''(x) + \dots</math>
+In this case, the three methods for approximating the derivative give us the following results:
+{| class="sortable" border="1"
+! Method !! Computed approximate value to <math>f'(x)</math> !! Error term (computed value minus actual value) !! Order of convergence (smallest exponent on <math>h</matH> with nonzero coefficient in Taylor expansion of error) -- higher order is better
+|-
+| forward [[difference quotient]] <math>\frac{f(x + h) - f(x)}{h}</math> || <math>f'(x) + \frac{h}{2}f''(x) + \frac{h^2}{6}f'''(x) + \dots</math> || <math>\frac{h}{2}f''(x) + \frac{h^2}{6}f'''(x) + \dots</math> || 1
+|-
+| backward [[difference quotient]] <math>\frac{f(x) - f(x - h)}{h}</math> || <math>f'(x) - \frac{h}{2}f''(x) + \frac{h^2}{6}f'''(x) + \dots</math> || <math>- \frac{h}{2}f''(x) + \frac{h^3}{6}f'''(x) + \dots</math> || 1
+|-
+| central difference quotient <math>\frac{f(x + (h/2)) - f(x - (h/2))}{h}</math> || <math>f'(x) + \frac{h^2}{12}f'''(x) + \dots</math> || <math>\frac{h^2}{12}f'''(x) + \dots</math> || 2
+|}
+We therefore see that the central difference quotient computes a substantially more precise value for the derivative.
+Note that we for the result above to hold, we do not require the function to have a Taylor series; we only need the function to be three or more times continuously differentiable (in fact, a somewhat weaker version holds if the function is only twice continuously differentiable).