Quadratic function of multiple variables

Definition

Consider variables $x_{1},x_{2},\dots ,x_{n}$. A quadratic function of the variables $x_{1},x_{2},\dots ,x_{n}$ is a function of the form:

$\left(\sum _{i=1}^{n}\sum _{j=1}^{n}a_{ij}x_{i}x_{j}\right)+\left(\sum _{i=1}^{n}b_{i}x_{i}\right)+c$

In vector form, if we denote by ${\vec {x}}$ the column vector with coordinates $x_{1},x_{2},\dots ,x_{n}$, then we can write the function as:

${\vec {x}}^{T}A{\vec {x}}+{\vec {b}}^{T}{\vec {x}}+c$

where $A$ is the $n\times n$ matrix with entries $a_{ij}$ and ${\vec {b}}$ is the column vector with entries $b_{i}$.

Key data

Item	Value
default domain	the whole of $\mathbb {R} ^{n}$
range	If the matrix $A$ is not positive semidefinite or negative semidefinite, the range is all of $\mathbb {R}$. If the matrix $A$ is positive semidefinite, the range is $[m,\infty )$ where $m$ is the minimum value. If the matrix $A$ is negative semidefinite, the range is $(-\infty ,m]$ where $m$ is the maximum value.

Cases

Positive definite case

First, we consider the case where $A$ is a positive definite matrix. In other words, we can write $A$ in the form:

$A=M^{T}M$

where $M$ is a $n\times n$ invertible matrix.

We can "complete the square" for this function:

$f({\vec {x}})=\left(M{\vec {x}}+{\frac {1}{2}}(M^{T})^{-1}{\vec {b}}\right)^{T}\left(M{\vec {x}}+{\frac {1}{2}}(M^{T})^{-1}{\vec {b}}\right)+\left(c-{\frac {1}{4}}{\vec {b}}^{T}M{\vec {b}}\right)$

In other words:

$f({\vec {x}})=\left\|M{\vec {x}}+{\frac {1}{2}}(M^{T})^{-1}{\vec {b}}\right\|^{2}+\left(c-{\frac {1}{4}}{\vec {b}}^{T}M{\vec {b}}\right)$

This is minimized when the expression whose norm we are measuring is zero, so that it is minimized when we have:

$M{\vec {x}}+{\frac {1}{2}}(M^{T})^{-1}{\vec {b}}={\vec {0}}$

Simplifying, we obtain that we minimum occurs at:

${\vec {x}}=-{\frac {1}{2}}A^{-1}{\vec {b}}$