Logistic function

Definition

The logistic function is a function with domain $\R$ and range the open interval $(0,1)$, defined as:

$x \mapsto \frac{1}{1 + e^{-x}}$

Equivalently, it can be written as:

$x \mapsto \frac{e^x}{e^x + 1}$

Yet another form that is sometimes used, because it makes some aspects of the symmetry more evident, is:

$x \mapsto \frac{e^{x/2}}{e^{x/2} + e^{-x/2}}$

For this page, we will denote the function by the letter $g$.

We may extend the logistic function to a function $[-\infty,\infty] \to [0,1]$, where $g(-\infty) = 0$ and $g(\infty) = 1$.

Probabilistic interpretation

The logistic function transforms the logarithm of the odds to the actual probability. Explicitly, given a probability $p$ (strictly between 0 and 1) of an event occurring, the odds in favor of $p$ are given as:

$\frac{p}{1 - p}$

This could take any value in $(0,\infty)$

The logarithm of odds is the expression:

$\ln\left(\frac{p}{1 - p}\right)$

If $x$ equals the above expression, then the function describing $p$ in terms of $x$ is the logistic function.

Key data

Item	Value
default domain	all of $\R$, i.e., all reals
range	the open interval $(0,1)$, i.e., the set $\{ x \mid 0 \le x \le 1 \}$
derivative	the derivative is $\frac{-e^{-x}}{(1 + e^{-x})^2}$. If we denote the logistic function by the letter $g$, then we can also write the derivative as $g'(x) = g(x)g(-x) = g(x)(1 - g(x))$
second derivative	If we denote the logistic function by the letter $g$, then we can also write the derivative as $g''(x) = g'(x)(1 - 2g(x)) = g(x)(1 - g(x))(1 - 2g(x)) = g(x)g(-x)(1 - 2g(x)) = g(x)g(-x)(g(-x) - g(x))$
logarithmic derivative	the logarithmic derivative is $\frac{e^{-x}}{1 + e^{-x}}$ If we denote the logistic function by $g$, the logarithmic derivative is $g(-x)$
antiderivative	the function $x \mapsto \ln(e^x + 1) + C = -\ln(g(-x)) + C$
critical points	none
critical points for the derivative (correspond to points of inflection for the function)	$x = 0$; the corresponding point on the graph of the function is $(0,1/2)$.
local maximal values and points of attainment	none
local minimum values and points of attainment	none
intervals of interest	increasing and concave up on $(-\infty,0)$ increasing and concave down on $(0,\infty)$
horizontal asymptotes	asymptote at $y = 0$ corresponding to the limit for $x \to -\infty$ asymptote at $y = 1$ corresponding to the limit for $x \to \infty$
inverse function	inverse logistic function or log-odds function given by $x \mapsto \ln \left(\frac{x}{1 - x} \right)$

Differentiation

First derivative

Consider the expression for $g(x)$:

$g(x) = \frac{1}{1 + e^{-x}} = (1 + e^{-x})^{-1}$

We can differentiate this using the chain rule for differentiation (the inner function being $x \mapsto 1 + e^{-x}$ and the outer function being the reciprocal function $t \mapsto 1/t$. We get:

$g'(x) = -(1 + e^{-x})^{-2}(-e^{-x})$

Simplifying, we get:

$g'(x) = \frac{e^{-x}}{(1 + e^{-x})^2}$

We can write this in an alternate way that is sometimes more useful. We split the expression as a product:

$g'(x) = \left( \frac{1}{1 + e^{-x}} \right) \left(\frac{e^{-x}}{1 + e^{-x}}\right)$

The first factor on the right is $g(x)$, and the second factor is $1 - g(x)$, so this simplifies to:

$g'(x) = g(x)(1 - g(x))$

Second derivative

Using the expression $g(x)(1 - g(x))$ for $g'$

From the above, we have:

$g'(x) = g(x) - (g(x))^2$

Differentiating both sides, we obtain:

$g''(x) = g'(x) - 2g(x)g'(x)$

This simplifies to:

$g''(x) = g'(x)(1 - 2g(x))$

We can now re-use the expression for $g'$ and obtain:

$g''(x) = g(x)(1 - g(x))(1 - 2g(x))$

Using the expression $g(x)g(-x)$ for $g'$

We have:

$g'(x) = g(x)g(-x)$

Using the product rule for differentiation and the chain rule for differentiation, we get:

$g''(x) = g'(x)g(-x) + (-g(x)g'(-x))$

Note from the expression that $g'$ shows that $g'$ is even, so we can rewrite $g'(-x)$ as $g'(x)$, and we get:

$g''(x) = g'(x)g(-x) - g'(x)g(x) = g'(x)(g(-x) - g(x))$

We can re-use the expression $g'(x) = g(x)g(-x)$ and obtain:

$g''(x) = g(x)g(-x)(g(-x) - g(x))$

Functional equations

Symmetry equation

The logistic function $g$ has the property that its graph $y = g(x)$ has symmetry about the point $(0,1/2)$. Explicitly, it satisfies the functional equation:

$g(x) + g(-x) = 1$

We can see this algebraically:

$g(-x) = \frac{1}{1 + e^{-(-x)}} = \frac{1}{1 + e^x}$

Multiply numerator and denominator by $e^{-x}$, and get:

$g(-x) = \frac{e^{-x}}{e^{-x} + 1} = 1 - \frac{1}{1 + e^{-x}} = 1 - g(x)$

Differential equation

As discussed in the #First derivative section, the logistic function satisfies the condition:

$g'(x) = g(x)(1 - g(x))$

Therefore, $y = g(x)$ is a solution to the autonomous differential equation:

$\frac{dy}{dx} = y(1 - y)$

The general solution to that equation is the function $y = g(x + C)$ where $C \in \R$. The initial condition $y = 1/2$ at $x = 0$ pinpoints the logistic function uniquely.

Points and intervals of interest

Critical points

The function has no critical points. To see this, note that the derivative is:

$g'(x) = g(x)(1 - g(x)) = \frac{e^{-x}}{(1 + e^{-x})^2}$

Note that the numerator is never zero, nor is the denominator. Therefore, the function is always defined and nonzero.

Intervals of increase and decrease

The derivative:

$g'(x) = g(x)(1 - g(x)) = \frac{e^{-x}}{(1 + e^{-x})^2}$

is always positive. So the function is increasing on all of $\R$.

The asymptotic values are:

$\lim_{x \to \infty} \frac{1}{1 + e^{-x}} = \frac{1}{1} = 1$

and:

$\lim_{x \to -\infty} \frac{1}{1 + e^{-x}} = \frac{1}{\to \infty} = 0$

In other words, the range of the function is the open interval $(0,1)$, and it increases throughout its domain.

Points of inflection

The second derivative is:

$g''(x) = g(x)(1 - g(x))(1 - 2g(x)) = g'(x)(1 - 2g(x))$

We already noted that $g'(x)$ is always defined and nonzero, so the only way for $g''(x)$ to be zero is if $1 - 2g(x) = 0$< or $g(x) = 1/2$. This solves to:

$\frac{1}{1 + e^{-x}} = \frac{1}{2}$

This solves to $e^{-x} = 1$, or $x = 0$.

Thus, the second derivative is 0 at the point $(0,1/2)$, i.e., with $x = 0$ and $g(x) = 1/2$.

Intervals of concave up and down

As above, we have:

$g''(x) = g'(x)(1 - 2g(x))$

We also noted that $g'(x) > 0$ for all $x$. Therefore, $g''(x) > 0$ for $x < 0$ and $g''(x) < 0$ for $x > 0$. Therefore, $g$ is:

• concave up for $x < 0$, i.e., $x \in (-\infty, 0)$
• concave down for $x > 0$, i.e., $x \in (0, \infty)$

Symmetry

We discussed above a functional equation satisfied by $g$:

$g(-x) = 1 - g(x)$

From this, the following can be deduced:

• The graph of $g$ has half-turn symmetry about the point $(0, 1/2)$.
• $g'$ is an even function. Note that this can also be seen from the actual expression: $g'(x) = g(x)g(-x)$. But we don't need the actual expression to deduce that it is even -- the functional equation above gives evenness.
• $g''$ is an odd function. This can be directly deduced from $g'$ being even, but can also be verified from the actual expression: $g''(x) = g'(x)(1 - 2g(x)) = g'(x)(g(-x) - g(x))$.

Integration

First antiderivative

Direct computation

We have:

$\int g(x) \, dx = \int \frac{1 \, dx}{1 + e^{-x}} = \int \frac{e^x \, dx}{e^x + 1} = \ln(e^x + 1) + C$

Computation in terms of functional equations for the logistic function

We have:

$g'(x) = g(x)(1 - g(x))$

We also have that $1 - g(x) = g(-x)$, so we get:

$g'(x) = g(x)g(-x)$

This can be rewritten as:

$\frac{d}{dx}(\ln(g(x)) = g(-x)$

By the chain rule for differentiation, we get:

$\frac{d}{dx}(\ln(g(-x)) = -g(x)$

Thus:

$\int g(x) \, dx = -\ln(g(-x)) + C$

This can be simplified and verified to be the same as the answer obtained by direct computation.