Logarithmic scoring rule
The logarithmic scoring rule is a scoring rule used to measure how well a given assignment of probabilities to values of a random variable performs on some real-world instances of the random variable. Explicitly, consider a random variable that can take distinct values . Suppose we estimate probabilities for these values respectively (with , ). The logarithmic scoring rule works as follows: for every instance of the random variable , we assign score equal to the negative of the logarithm of the corresponding probability . Explicitly, if the instances are , the total score is:
Note that the base of the logarithms does not matter: it could be any fixed number greater than 1.
The smaller the value of the score with the logarithmic scoring rule, the better the assignment of probabilities has performed according to the rule.
- Logarithmic scoring rule is proper: In expectation, if the actual probabilities of the values are respectively, then the choice of values that minimizes our expected score is .
Relation with logarithmic loss functions in logistic regression
The logarithmic loss function used in logistic regression relies on the logarithmic scoring rule, but with the following twist: the probabilities are not fixed numbers, but themselves depend on parameters that vary with the random instances that we are presented.