Logistic log-loss function of one variable

Definition

The logistic log-loss function of one variable is obtained by composing the logarithmic cost function with the logistic function, and it is of importance in the analysis of logistic regression.

Explicitly, the function has the form:

$f(x)=-(p\ln (g(x))+(1-p)\ln (1-g(x)))$

where $g$ is the logistic function and $\ln$ denotes the natural logarithm. Explicitly, $g(x)=\frac{1}{1+e^{-x}}$.

Note that $1-g(x)=g(-x)$, so the above can be written as:

$f(x)=-(p\ln (g(x))+(1-p)\ln (g(-x)))$

We restrict $p$ to the interval $[0,1]$. Conceptually, $p$ is the corresponding probability.

More explicitly, $f$ is the function:

$f(x)=p\ln (1+e^{-x})+(1-p)\ln (1+e^x)$

Key data

Item	Value
default domain	all of $\mathbb{R}$, i.e., all reals
range	$[m,\mathrm{\infty })$ where $m$ is the minimum value, given as $-(p\ln p+(1-p)\ln (1-p))$.
local maximum value and points of attainment	No local maximum values
local minimum value and points of attainment	Local minimum value $-(p\ln p+(1-p)\ln (1-p))$, attained at $x=\ln (\frac{p}{1-p})$.
derivative	$g(x)-p$ where $g$ is the logistic function
second derivative	$g(x)(1-g(x))=g(x)g(-x)$
third derivative	$g(x)g(-x)(g(x)-g(-x))=g(x)(1-g(x))(1-2g(x))$

Differentiation

WHAT WE USE: chain rule for differentiation, Logistic function#First derivative

First derivative

We use that:

$g^{\prime }(x)=g(x)(1-g(x))=g(x)g(-x)$

or equivalently:

$\frac{d}{dx}(\ln (g(x))=1-g(x)=g(-x)$

Similarly:

$\frac{d}{dx}(\ln (g(-x))=-g(x)$

Plugging these in, we get:

$f^{\prime }(x)=-(p(1-g(x))+(1-p)(-g(x)))$

This simplifies to:

$f^{\prime }(x)=g(x)-p$

Second derivative

Using the first derivative and the expression for $g^{\prime }$, we obtain:

$f^{″}(x)=g(x)(1-g(x))=g(x)g(-x)$

Note that the second derivative is independent of $p$.