Logistic log-loss function of one variable

Definition

The logistic log-loss function of one variable is obtained by composing the logarithmic cost function with the logistic function, and it is of importance in the analysis of logistic regression.

Explicitly, the function has the form:

$f(x)=-(p\ln (g(x))+(1-p)\ln (1-g(x)))$

where $g$ is the logistic function and $\ln$ denotes the natural logarithm. Explicitly, $g(x)=\frac{1}{1+e^{-x}}$.

Note that $1-g(x)=g(-x)$, so the above can be written as:

$f(x)=-(p\ln (g(x))+(1-p)\ln (g(-x)))$

We restrict $p$ to the interval $[0,1]$. Conceptually, $p$ is the corresponding probability.

More explicitly, $f$ is the function:

$f(x)=p\ln (1+e^{-x})+(1-p)\ln (1+e^x)$

Differentiation

WHAT WE USE: chain rule for differentiation, Logistic function#First derivative

First derivative

We use that:

$g^{\prime }(x)=g(x)(1-g(x))=g(x)g(-x)$

or equivalently:

$\frac{d}{dx}(\ln (g(x))=1-g(x)=g(-x)$

Similarly:

$\frac{d}{dx}(\ln (g(-x))=-g(x)$

Plugging these in, we get:

$f^{\prime }(x)=-(p(1-g(x))+(1-p)(-g(x)))$

This simplifies to:

$f^{\prime }(x)=g(x)-p$

Second derivative

Using the first derivative and the expression for $g^{\prime }$, we obtain:

$f^{″}(x)=g(x)(1-g(x))=g(x)g(-x)$