Quadratic function of multiple variables

Definition

Consider variables $x_1,x_2,\dots ,x_n$. A quadratic function of the variables $x_1,x_2,\dots ,x_n$ is a function of the form:

$(\sum _{i=1}^n\sum _{j=1}^n{a}_{ij}{x}_i{x}_j)+(\sum _{i=1}^n{b}_i{x}_i)+c$

In vector form, if we denote by $\overrightarrow{x}$ the column vector with coordinates $x_1,x_2,\dots ,x_n$, then we can write the function as:

${\overrightarrow{x}}^{T}A\overrightarrow{x}+{\overrightarrow{b}}^T\overrightarrow{x}+c$

where $A$ is the $n\times n$ matrix with entries $a_{ij}$ and $\overrightarrow{b}$ is the column vector with entries $b_i$.

Key data

Item	Value
default domain	the whole of ${\mathbb{R}}^n$
range	If the matrix $A$ is not positive semidefinite or negative semidefinite, the range is all of $\mathbb{R}$. If the matrix $A$ is positive semidefinite, the range is $[m,\mathrm{\infty })$ where $m$ is the minimum value. If the matrix $A$ is negative semidefinite, the range is $(-\mathrm{\infty },m]$ where $m$ is the maximum value.

Cases

Positive definite case

First, we consider the case where $A$ is a positive definite matrix. In other words, we can write $A$ in the form:

$A=M^{T}M$

where $M$ is a $n\times n$ invertible matrix.

We can "complete the square" for this function:

$f(\overrightarrow{x})={(M\overrightarrow{x}+\frac{1}{2}(M^T{)}^{-1}\overrightarrow{b})}^T(M\overrightarrow{x}+\frac{1}{2}(M^T{)}^{-1}\overrightarrow{b})+(c-\frac{1}{4}{\overrightarrow{b}}^{T}M\overrightarrow{b})$

In other words:

$f(\overrightarrow{x})={\Vert M\overrightarrow{x}+\frac{1}{2}(M^T{)}^{-1}\overrightarrow{b}\Vert }^2+(c-\frac{1}{4}{\overrightarrow{b}}^{T}M\overrightarrow{b})$

This is minimized when the expression whose norm we are measuring is zero, so that it is minimized when we have:

$M\overrightarrow{x}+\frac{1}{2}(M^T{)}^{-1}\overrightarrow{b}=\overrightarrow{0}$

Simplifying, we obtain that we minimum occurs at:

$\overrightarrow{x}=-\frac{1}{2}{A}^{-1}\overrightarrow{b}$