Hessian matrix - Revision history

Vipul at 00:06, 24 April 2014

2014-04-24T00:06:55Z

← Older revision		Revision as of 00:06, 24 April 2014
Line 34:		Line 34:
	In symbols, for a function <math>f</math> of variables <math>x_1,x_2,\dots,x_n</math>, we get:		In symbols, for a function <math>f</math> of variables <math>x_1,x_2,\dots,x_n</math>, we get:

	<math>H(f)_{ij} = f_{~~X_ix_j~~} = f_{x_jx_i} = H(f)_{ji} \ \forall i,j \in \{ 1,2,\dots,n \}</math>		<math>H(f)_{ij} = f_{x_ix_j} = f_{x_jx_i} = H(f)_{ji} \ \forall i,j \in \{ 1,2,\dots,n \}</math>

	==Relation with second-order directional derivatives==		==Relation with second-order directional derivatives==

Vipul: /* Interpretation as second derivative */

2013-03-07T21:41:09Z

Interpretation as second derivative

← Older revision		Revision as of 21:41, 7 March 2013
Line 18:		Line 18:
	Thus, the Hessian matrix is the correct notion of second derivative.		Thus, the Hessian matrix is the correct notion of second derivative.

	<center>{{#widget:YouTube\|id=~~zlWJforCpN4~~}}</center>		<center>{{#widget:YouTube\|id=X84tMY_7hqU}}</center>

	==Relation with second-order partial derivatives==		==Relation with second-order partial derivatives==

Vipul at 02:33, 13 May 2012

2012-05-13T02:33:34Z

← Older revision		Revision as of 02:33, 13 May 2012
Line 17:		Line 17:

	Thus, the Hessian matrix is the correct notion of second derivative.		Thus, the Hessian matrix is the correct notion of second derivative.

			<center>{{#widget:YouTube\|id=zlWJforCpN4}}</center>

	==Relation with second-order partial derivatives==		==Relation with second-order partial derivatives==

Vipul: /* Relation with second-order directional derivatives */

2012-05-12T16:54:40Z

Relation with second-order directional derivatives

← Older revision		Revision as of 16:54, 12 May 2012
Line 43:		Line 43:

	where <math>\overline{u},\overline{v}</math> are treated as column vectors, so <math>\overline{u}^T</math> is <math>\overline{u}</math> as a row vector, and <math>\overline{v}</math> is <math>\overline{v}</math> as a column vector. The multiplication on the right side is matrix multiplication. Note that this tells us that the bilinear form corresponding to the Hessian matrix outputs second-order directional derivatives.		where <math>\overline{u},\overline{v}</math> are treated as column vectors, so <math>\overline{u}^T</math> is <math>\overline{u}</math> as a row vector, and <math>\overline{v}</math> is <math>\overline{v}</math> as a column vector. The multiplication on the right side is matrix multiplication. Note that this tells us that the bilinear form corresponding to the Hessian matrix outputs second-order directional derivatives.

			Note further that if the second-order mixed partials are continuous, this forces the Hessian matrix to be symmetric, which means that the bilinear form we obtain is symmetric, and hence, we will get:

			<math>D_{\overline{v}}(D_{\overline{u}}(f)) = D_{\overline{u}}(D_{\overline{v}}(f))</math>

Vipul: /* Definition */

2012-05-12T16:53:15Z

Definition

← Older revision		Revision as of 16:53, 12 May 2012
Line 5:		Line 5:
	===Definition in terms of Jacobian matrix and gradient vector===		===Definition in terms of Jacobian matrix and gradient vector===

	Suppose <math>f</math> is a real-valued function of <math>n</math> variables <math>x_1,x_2,\dots,x_n</math>. The '''Hessian matrix'' of <math>f</math> is a <math>n \times n</math>-matrix-valued function with [[domain]] a subset of the domain of <math>f</math>, defined as follows: the Hessian matrix at any point in the domain is the [[Jacobian matrix]] of the [[gradient vector]] of <math>f</math> at the point. In point-free notation, we denote by <math>H(f)</math> the Hessian matrix function, and we define it as:		Suppose <math>f</math> is a real-valued function of <math>n</math> variables <math>x_1,x_2,\dots,x_n</math>. The '''Hessian matrix''' of <math>f</math> is a <math>n \times n</math>-matrix-valued function with [[domain]] a subset of the domain of <math>f</math>, defined as follows: the Hessian matrix at any point in the domain is the [[Jacobian matrix]] of the [[gradient vector]] of <math>f</math> at the point. In point-free notation, we denote by <math>H(f)</math> the Hessian matrix function, and we define it as:

	<math>H(f) = J(\nabla f)</math>		<math>H(f) = J(\nabla f)</math>

Vipul at 16:52, 12 May 2012

2012-05-12T16:52:50Z

Show changes

Vipul: /* Definition in terms of second-order partial derivatives */

2012-05-12T16:15:57Z

Definition in terms of second-order partial derivatives

← Older revision		Revision as of 16:15, 12 May 2012
Line 23:		Line 23:

	Wherever the Hessian matrix for a function exists, its entries can be described as second-order partial derivatives of the function. Explicitly, for a function <math>f</math> is a real-valued function of <math>n</math> variables <math>x_1,x_2,\dots,x_n</math>, the Hessian matrix <math>H(f)</math> is a <math>n \times n</math>-matrix-valued function whose <math>(ij)^{th}</math> entry is the second-order partial derivative <math>\partial^2f/(\partial x_j\partial x_i)</math>, which is the same as <math>f_{x_ix_j}</math>. Note that the diagonal entries give second-order pure partial derivatives whereas the off-diagonal entries give [[second-order mixed partial derivative]]s.		Wherever the Hessian matrix for a function exists, its entries can be described as second-order partial derivatives of the function. Explicitly, for a function <math>f</math> is a real-valued function of <math>n</math> variables <math>x_1,x_2,\dots,x_n</math>, the Hessian matrix <math>H(f)</math> is a <math>n \times n</math>-matrix-valued function whose <math>(ij)^{th}</math> entry is the second-order partial derivative <math>\partial^2f/(\partial x_j\partial x_i)</math>, which is the same as <math>f_{x_ix_j}</math>. Note that the diagonal entries give second-order pure partial derivatives whereas the off-diagonal entries give [[second-order mixed partial derivative]]s.

			Some people choose to ''define'' the Hessian matrix as the matrix whose entries are the second-order partial derivatives as indicated here. However, that is not quite the correct definition of Hessian matrix because it is possible for all the second-order partial derivatives to exist but for the function to not be twice differentiable at the point. The main disadvantage of defining the Hessian matrix in the more expansive sense (i.e., in terms of second-order partial derivatives) is that all the important results about the Hessian matrix crucially rely on the function being twice differentiable, so we don't actually gain anything by using the more expansive definition.

	==Computationally useful definition at a point==		==Computationally useful definition at a point==

Vipul: /* Definition in terms of second-order partial derivatives */

2012-05-12T16:12:59Z

Definition in terms of second-order partial derivatives

← Older revision		Revision as of 16:12, 12 May 2012
Line 22:		Line 22:
	{{further\|[[Relation between Hessian matrix and second-order partial derivatives]]}}		{{further\|[[Relation between Hessian matrix and second-order partial derivatives]]}}

	Wherever the Hessian matrix for a function exists, its entries can be described as second-order partial derivatives of the function. Explicitly, for a function <math>f</math> is a real-valued function of <math>n</math> variables <math>x_1,x_2,\dots,x_n</math>, the Hessian matrix <math>H(f)</math> is a <math>n \times n</math>-matrix-valued function whose <math>(ij)^{th}</math> entry is the second-order partial derivative <math>\partial^2/(\partial x_j\partial x_i}</math>, which is the same as <math>f_{x_ix_j}</math>. Note that the diagonal entries give second-order pure partial derivatives whereas the off-diagonal entries give [[second-order mixed partial derivative]]s.		Wherever the Hessian matrix for a function exists, its entries can be described as second-order partial derivatives of the function. Explicitly, for a function <math>f</math> is a real-valued function of <math>n</math> variables <math>x_1,x_2,\dots,x_n</math>, the Hessian matrix <math>H(f)</math> is a <math>n \times n</math>-matrix-valued function whose <math>(ij)^{th}</math> entry is the second-order partial derivative <math>\partial^2f/(\partial x_j\partial x_i)</math>, which is the same as <math>f_{x_ix_j}</math>. Note that the diagonal entries give second-order pure partial derivatives whereas the off-diagonal entries give [[second-order mixed partial derivative]]s.

	==Computationally useful definition at a point==		==Computationally useful definition at a point==

Vipul at 16:12, 12 May 2012

2012-05-12T16:12:23Z

@@ Line 1: / Line 1: @@
 {{multivariable analogue of|second derivative}}
 ==Definition at a point==
 ===For a function of two variables at a point===
-Suppose <math>f</math> is a real-valued function of two variables <math>x,y</math> and <math>(x_0,y_0)</math> is a point in the domain of <math>f</math>. Suppose all the four second-order partial derivatives exist at <math>(x_0,y_0)</math>, i.e., the two pure second-order partials <math>f_{xx}(x_0,y_0),f_{yy}(x_0,y_0)</math> exist, and so do the two [[second-order mixed partial derivative]]s <math>f_{xy}(x_0,y_0)</math> and <math>f_{yx}(x_0,y_0)</math>. Then, the Hessian matrix of <math>f</math> at <math>(x_0,y_0)</math>, denoted <math>H(f)(x_0,y_0)</math>, is a <math>2 \times 2</math> matrix of real numbers defined as follows:
+Suppose <math>f</math> is a real-valued function of two variables <math>x,y</math> and <math>(x_0,y_0)</math> is a point in the domain of <math>f</math> at which <math>f</matH> is twice differentiable. In particular, this means that all the four second-order partial derivatives exist at <math>(x_0,y_0)</math>, i.e., the two pure second-order partials <math>f_{xx}(x_0,y_0),f_{yy}(x_0,y_0)</math> exist, and so do the two [[second-order mixed partial derivative]]s <math>f_{xy}(x_0,y_0)</math> and <math>f_{yx}(x_0,y_0)</math>. Then, the Hessian matrix of <math>f</math> at <math>(x_0,y_0)</math>, denoted <math>H(f)(x_0,y_0)</math>, can be expressed explicitly as a <math>2 \times 2</math> matrix of real numbers defined as follows:
 <math>\begin{pmatrix} f_{xx}(x_0,y_0) & f_{xy}(x_0,y_0) \\ f_{yx}(x_0,y_0) & f_{yy}(x_0,y_0) \\\end{pmatrix}</math>
@@ Line 12: / Line 36: @@
 ===For a function of multiple variables at a point===
-Suppose <math>f</math> is a real-valued function of multiple variables <math>(x_1,x_2,\dots,x_n)</math>. Suppose <math>(a_1,a_2,\dots,a_n)</math> is a point in the domain of <math>f</math>. In other words, <math>a_1,a_2,\dots,a_n</math> are real numbers and the point has coordinates <math>x_1 = a_1, x_2 = a_2, \dots,x_n = a_n</math>. Suppose, further, that all the second-order partials (pure and mixed) of <math>f</math> with respect to these variables exist at the point <math>(a_1,a_2,\dots,a_n)</math>. Then, the Hessian matrix of <math>f</math> at <math>(a_1,a_2,\dots,a_n)</math>, denoted <math>H(f)(a_1,a_2,\dots,a_n)</math>, is a <math>n \times n</math> matrix of real numbers defined as follows:
+Suppose <math>f</math> is a real-valued function of multiple variables <math>(x_1,x_2,\dots,x_n)</math>. Suppose <math>(a_1,a_2,\dots,a_n)</math> is a point in the domain of <math>f</math> at which <math>f</math> is twice differentiable. In other words, <math>a_1,a_2,\dots,a_n</math> are real numbers and the point has coordinates <math>x_1 = a_1, x_2 = a_2, \dots,x_n = a_n</math>. Suppose, further, that all the second-order partials (pure and mixed) of <math>f</math> with respect to these variables exist at the point <math>(a_1,a_2,\dots,a_n)</math>. Then, the Hessian matrix of <math>f</math> at <math>(a_1,a_2,\dots,a_n)</math>, denoted <math>H(f)(a_1,a_2,\dots,a_n)</math>, is a <math>n \times n</math> matrix of real numbers that can be expressed explicitly as follows:
 The <math>(ij)^{th}</math> entry (i.e., the entry in the <math>i^{th}</math> row and <math>j^{th}</math> column) is <math>f_{x_ix_j}(a_1,a_2,\dots,a_n)</math>. This is the same as <math>\frac{\partial^2}{\partial x_j \partial x_i}f(x_1,x_2,\dots,x_n)|_{(x_1,x_2,\dots,x_n) = (a_1,a_2,\dots,a_n)}</math>. Note that in the two notations, the order in which we write the partials differs because the convention differs (left-to-right versus right-to-left).

Vipul at 15:34, 4 May 2012

2012-05-04T15:34:50Z

← Older revision		Revision as of 15:34, 4 May 2012
Line 1:		Line 1:
			{{multivariable analogue of\|second derivative}}
	==Definition at a point==		==Definition at a point==