Logarithmic scoring rule is the only proper scoring rule up to affine transformations in case of more than two classes

Statement

Let $f$ be a proper scoring rule for a binary classification problem into $n$ classes, $n\geq 3$.

Explicitly, consider a random variable $X$ that can take $n$ distinct values $1,2,\dots ,n$. Suppose we estimate probabilities $p_{1},p_{2},\dots ,p_{n}$ for these values respectively (with $p_{i}\in [0,1]$, $\sum _{i=1}^{n}p_{i}=1$). The logarithmic scoring rule works as follows: for every instance of the random variable $X$, we assign score of $f(p_{i})$. Explicitly, if the instances are $X_{1},X_{2},\dots ,X_{m}$, the total score is:

$\sum _{j=1}^{m}f(p_{X_{j}})$

$f$ is required to satisfy the condition that, if the actual probabilities are $q_{1},q_{2},\dots ,q_{n}$, then the assignment that minimizes the expected value of the score is $p_{1}=q_{1},p_{2}=q_{2},\dots ,p_{n}=q_{n}$.

Then, we must have $f$ of the form:

$f(p)=-A\log p+B$

where $A>0$ and $B$ is any real number. In other words, $f$ must be the same as the logarithmic scoring rule, up to affine transformations.

Related facts

• Logarithmic scoring rule is proper