Hessian matrix

Definition at a point

For a function of two variables at a point

Suppose $f$ is a real-valued function of two variables $x,y$ and $(x_{0},y_{0})$ is a point in the domain of $f$. Suppose all the four second-order partial derivatives exist at $(x_{0},y_{0})$, i.e., the two pure second-order partials $f_{xx}(x_{0},y_{0}),f_{yy}(x_{0},y_{0})$ exist, and so do the two second-order mixed partial derivatives $f_{xy}(x_{0},y_{0}$ and $f_{yx}(x_{0},y_{0})$. Then, the Hessian matrix of $f$ at $(x_{0},y_{0})$, denoted $H(f)(x_{0},y_{0})$, is a $2\times 2$ matrix of real numbers defined as follows:

${\begin{pmatrix}f_{xx}(x_{0},y_{0})&f_{xy}(x_{0},y_{0})\\f_{yx}(x_{0},y_{0})&f_{yy}(x_{0},y_{0})\\\end{pmatrix}}$

For a function of multiple variables at a point

Suppose $f$ is a real-valued function of multiple variables $(x_{1},x_{2},\dots ,x_{n})$. Suppose $(a_{1},a_{2},\dots ,a_{n})$ is a point in the domain of $f$. In other words, $a_{1},a_{2},\dots ,a_{n}$ are real numbers and the point has coordinates $x_{1}=a_{1},x_{2}=a_{2},\dots ,x_{n}=a_{n}$. Suppose, further, that all the second-order partials (pure and mixed) of $f$ with respect to these variables exist at the point $(a_{1},a_{2},\dots ,a_{n})$. Then, the Hessian matrix of $f$ at $(a_{1},a_{2},\dots ,a_{n})$, denoted $H(f)(a_{1},a_{2},\dots ,a_{n})$, is a $n\times n$ matrix of real numbers defined as follows:

The $(ij)^{th}$ entry (i.e., the entry in the $i^{th}$ row and $j^{th}$ column) is $f_{x_{i}x_{j}}(a_{1},a_{2},\dots ,a_{n})$. This is the same as ${\frac {\partial ^{2}f}{\partial x_{j}\partial x_{i}}}f(a_{1},a_{2},\dots ,a_{n})$. Note that in the two notations, the order in which we write the partials differs because the convention differs (left-to-right versus right-to-left).

The matrix looks like this:

Failed to parse (unknown function "\begin{pmatrix}"): {\displaystyle \begin{pmatrix} f_{x_1x_1}(a_1,a_2,\dots,a_n) & f_{x_1x_2}(a_1,a_2,\dots,a_n) & \dots & f_{x_2x_n}(a_1,a_2,\dots,a_n)\\ f_{x_2x_1}(a_1,a_2,\dots,a_n) & f_{x_2x_2}(a_1,a_2,\dots,a_n) & \dots & f_{x_2x_n}(a_1,a_2,\dots,a_n)\\ \dot & \dot & \dot & \dot\\ \dot & \dot & \dot & \dot\\ \dot & \dot & \dot & \dot\\ f_{x_nx_1}(a_1,a_2,\dots,a_n) & f_{x_nx_2}(a_1,a_2,\dots,a_n) & \dots & f_{x_nx_n}(a_1,a_2,\dots,a_n)\\\end{pmatrix}}

Definition as a function

For a function of two variables

Suppose $f$ is a real-valued function of two variables $x,y$. The Hessian matrix of $f$, denoted $H(f)$, is a $2\times 2$ matrix-valued function that sends each point to the Hessian matrix at that point, if that matrix is defined. It is defined as:

$(x_{0},y_{0})\mapsto H(f)(x_{0},y_{0})={\begin{pmatrix}f_{xx}(x_{0},y_{0})&f_{xy}(x_{0},y_{0})\\f_{yx}(x_{0},y_{0})&f_{yy}(x_{0},y_{0})\\\end{pmatrix}}$

In the point-free notation, we can write this as:

$H(f)={\begin{pmatrix}f_{xx}&f_{xy}\\f_{yx}&f_{yy}\\\end{pmatrix}}$

Under continuity assumptions

If we assume that all the second-order partials of $f$ are continuous functions everywhere, then the following happens:

• The Hessian matrix of $f$ at any point is a symmetric matrix, i.e., its $(ij)^{th}$ entry equals its $(ji)^{th}$ entry. This follows from Clairaut's theorem on equality of mixed partials.
• $f$ is twice differentiable as a function. Hence, the Hessian matrix of $f$ is the same as the Jacobian matrix of the gradient vector $\nabla f$, where the latter is viewed as a vector-valued function.

Note that the second conclusion actually only requires the existence of the gradient vector, hence it holds even if the second-order partials are not continuous.