Logistic function

From Calculus
Jump to: navigation, search

Definition

The logistic function is a function with domain \R and range the open interval (0,1), defined as:

x \mapsto \frac{1}{1 + e^{-x}}

Equivalently, it can be written as:

x \mapsto \frac{e^x}{e^x + 1}

Yet another form that is sometimes used, because it makes some aspects of the symmetry more evident, is:

x \mapsto \frac{e^{x/2}}{e^{x/2} + e^{-x/2}}

For this page, we will denote the function by the letter g.

We may extend the logistic function to a function [-\infty,\infty] \to [0,1], where g(-\infty) = 0 and g(\infty) = 1.

Probabilistic interpretation

The logistic function transforms the logarithm of the odds to the actual probability. Explicitly, given a probability p (strictly between 0 and 1) of an event occurring, the odds in favor of p are given as:

\frac{p}{1 - p}

This could take any value in (0,\infty)

The logarithm of odds is the expression:

\ln\left(\frac{p}{1 - p}\right)

If x equals the above expression, then the function describing p in terms of x is the logistic function.

Key data

Item Value
default domain all of \R, i.e., all reals
range the open interval (0,1), i.e., the set \{ x \mid 0 \le x \le 1 \}
derivative the derivative is \frac{-e^{-x}}{(1 + e^{-x})^2}.
If we denote the logistic function by the letter g, then we can also write the derivative as g'(x) = g(x)g(-x) = g(x)(1 - g(x))
second derivative If we denote the logistic function by the letter g, then we can also write the derivative as g''(x) = g'(x)(1 - 2g(x)) = g(x)(1 - g(x))(1 - 2g(x)) = g(x)g(-x)(1 - 2g(x)) = g(x)g(-x)(g(-x) - g(x))
logarithmic derivative the logarithmic derivative is \frac{e^{-x}}{1 + e^{-x}}
If we denote the logistic function by g, the logarithmic derivative is g(-x)
antiderivative the function x \mapsto \ln(e^x + 1) + C = -\ln(g(-x)) + C
critical points none
critical points for the derivative (correspond to points of inflection for the function) x = 0; the corresponding point on the graph of the function is (0,1/2).
local maximal values and points of attainment none
local minimum values and points of attainment none
intervals of interest increasing and concave up on (-\infty,0)
increasing and concave down on (0,\infty)
horizontal asymptotes asymptote at y = 0 corresponding to the limit for x \to -\infty
asymptote at y = 1 corresponding to the limit for x \to \infty
inverse function inverse logistic function or log-odds function given by x \mapsto \ln \left(\frac{x}{1 - x} \right)

Differentiation

First derivative

Consider the expression for g(x):

g(x) = \frac{1}{1 + e^{-x}} = (1 + e^{-x})^{-1}

We can differentiate this using the chain rule for differentiation (the inner function being x \mapsto 1 + e^{-x} and the outer function being the reciprocal function t \mapsto 1/t. We get:

g'(x) = -(1 + e^{-x})^{-2}(-e^{-x})

Simplifying, we get:

g'(x) = \frac{e^{-x}}{(1 + e^{-x})^2}

We can write this in an alternate way that is sometimes more useful. We split the expression as a product:

g'(x) = \left( \frac{1}{1 + e^{-x}} \right) \left(\frac{e^{-x}}{1 + e^{-x}}\right)

The first factor on the right is g(x), and the second factor is 1 - g(x), so this simplifies to:

g'(x) = g(x)(1 - g(x))

Second derivative

Using the expression g(x)(1 - g(x)) for g'

From the above, we have:

g'(x) = g(x) - (g(x))^2

Differentiating both sides, we obtain:

g''(x) = g'(x) - 2g(x)g'(x)

This simplifies to:

g''(x) = g'(x)(1 - 2g(x))

We can now re-use the expression for g' and obtain:

g''(x) = g(x)(1 - g(x))(1 - 2g(x))

Using the expression g(x)g(-x) for g'

We have:

g'(x) = g(x)g(-x)

Using the product rule for differentiation and the chain rule for differentiation, we get:

g''(x) = g'(x)g(-x) + (-g(x)g'(-x))

Note from the expression that g' shows that g' is even, so we can rewrite g'(-x) as g'(x), and we get:

g''(x) = g'(x)g(-x) - g'(x)g(x) = g'(x)(g(-x) - g(x))

We can re-use the expression g'(x) = g(x)g(-x) and obtain:

g''(x) = g(x)g(-x)(g(-x) - g(x))

Functional equations

Symmetry equation

The logistic function g has the property that its graph y = g(x) has symmetry about the point (0,1/2). Explicitly, it satisfies the functional equation:

g(x) + g(-x) = 1

We can see this algebraically:

g(-x) = \frac{1}{1 + e^{-(-x)}} = \frac{1}{1 + e^x}

Multiply numerator and denominator by e^{-x}, and get:

g(-x) = \frac{e^{-x}}{e^{-x} + 1} = 1 - \frac{1}{1 + e^{-x}} = 1 - g(x)

Differential equation

As discussed in the #First derivative section, the logistic function satisfies the condition:

g'(x) = g(x)(1 - g(x))

Therefore, y = g(x) is a solution to the autonomous differential equation:

\frac{dy}{dx} = y(1 - y)

The general solution to that equation is the function y = g(x + C) where C \in \R. The initial condition y = 1/2 at x = 0 pinpoints the logistic function uniquely.

Points and intervals of interest

Critical points

The function has no critical points. To see this, note that the derivative is:

g'(x) = g(x)(1 - g(x)) = \frac{e^{-x}}{(1 + e^{-x})^2}

Note that the numerator is never zero, nor is the denominator. Therefore, the function is always defined and nonzero.

Intervals of increase and decrease

The derivative:

g'(x) = g(x)(1 - g(x)) = \frac{e^{-x}}{(1 + e^{-x})^2}

is always positive. So the function is increasing on all of \R.

The asymptotic values are:

\lim_{x \to \infty} \frac{1}{1 + e^{-x}} = \frac{1}{1} = 1

and:

\lim_{x \to -\infty} \frac{1}{1 + e^{-x}} = \frac{1}{\to \infty} = 0

In other words, the range of the function is the open interval (0,1), and it increases throughout its domain.

Points of inflection

The second derivative is:

g''(x) = g(x)(1 - g(x))(1 - 2g(x)) = g'(x)(1 - 2g(x))

We already noted that g'(x) is always defined and nonzero, so the only way for g''(x) to be zero is if 1 - 2g(x) = 0< or g(x) = 1/2. This solves to:

\frac{1}{1 + e^{-x}} = \frac{1}{2}

This solves to e^{-x} = 1, or x = 0.

Thus, the second derivative is 0 at the point (0,1/2), i.e., with x = 0 and g(x) = 1/2.

Intervals of concave up and down

As above, we have:

g''(x) = g'(x)(1 - 2g(x))

We also noted that g'(x) > 0 for all x. Therefore, g''(x) > 0 for x < 0 and g''(x) < 0 for x > 0. Therefore, g is:

  • concave up for x < 0, i.e., x \in (-\infty, 0)
  • concave down for x > 0, i.e., x \in (0, \infty)

Symmetry

We discussed above a functional equation satisfied by g:

g(-x) = 1 - g(x)

From this, the following can be deduced:

  • The graph of g has half-turn symmetry about the point (0, 1/2).
  • g' is an even function. Note that this can also be seen from the actual expression: g'(x) = g(x)g(-x). But we don't need the actual expression to deduce that it is even -- the functional equation above gives evenness.
  • g'' is an odd function. This can be directly deduced from g' being even, but can also be verified from the actual expression: g''(x) = g'(x)(1 - 2g(x)) = g'(x)(g(-x) - g(x)).

Integration

First antiderivative

Direct computation

We have:

\int g(x) \, dx = \int \frac{1 \, dx}{1 + e^{-x}} = \int \frac{e^x \, dx}{e^x + 1} = \ln(e^x + 1) + C

Computation in terms of functional equations for the logistic function

We have:

g'(x) = g(x)(1 - g(x))

We also have that 1 - g(x) = g(-x), so we get:

g'(x) = g(x)g(-x)

This can be rewritten as:

\frac{d}{dx}(\ln(g(x)) = g(-x)

By the chain rule for differentiation, we get:

\frac{d}{dx}(\ln(g(-x)) = -g(x)

Thus:

\int g(x) \, dx = -\ln(g(-x)) + C

This can be simplified and verified to be the same as the answer obtained by direct computation.