Likelihood function

Lore & Background

The likelihood function is defined as any function of a parameter θ equal to c times the probability (or probability density) of the data x given θ, for some positive constant c. For discrete distributions, it is the probability mass function evaluated at the observed data, viewed as a function of the parameter. For continuous distributions, it is the probability density function similarly reoriented. The notation L(θ | x) is commonly used, and the function is not a probability density over θ—a common misinterpretation with potentially serious consequences, such as the prosecutor's fallacy.

Reader's Guide

The likelihood function serves as the basis for maximum likelihood estimation, where the parameter value that maximizes the likelihood is used as a point estimate for the unknown parameter. The Fisher information, often approximated by the Hessian matrix of the likelihood at its maximum, indicates the precision of that estimate. In Bayesian statistics, the likelihood is combined with a prior distribution via Bayes' rule to obtain the posterior probability of the parameter given the data. The likelihood itself does not give the probability that a parameter value is true; it only reflects the probability of observing the data under that parameter. This distinction is critical for correct statistical reasoning.

Did You Know?

• The likelihood function is defined as c times the probability of data given a parameter, for any positive constant c.
• In maximum likelihood estimation, the parameter that maximizes the likelihood serves as a point estimate.
• The Fisher information, often approximated by the likelihood's Hessian matrix at the maximum, indicates estimate precision.
• The likelihood function should not be confused with the posterior probability of the parameter given the data.