Covariance

Lore & Background

Covariance is defined for two jointly distributed real-valued random variables with finite second moments as the expected value of the product of their deviations from their individual expected values. This is expressed as cov(X,Y) = E[(X - E[X])(Y - E[Y])], which simplifies to E[XY] - E[X]E[Y]. The units of covariance are the product of the units of the two variables, so changing units (e.g., from meters to millimeters) changes the covariance value proportionally.

Reader's Guide

Covariance serves as a fundamental concept in statistics, providing a measure of how two random variables change together. Its sign indicates the tendency of the linear relationship: positive when variables show similar behavior, negative when they show opposite behavior. However, because covariance has units and its magnitude is affected by those units, it is difficult to assess the strength of the relationship from covariance alone. To compare the strength of joint association between different pairs of random variables with possibly different units, the correlation coefficient is used, which normalizes covariance to a value between -1 and 1 by dividing by the product of the standard deviations. A distinction exists between the population covariance, a parameter of the joint probability distribution, and the sample covariance, which serves as both a sample descriptor and an estimate of the population parameter. The identity cov(X,Y) = E[XY] - E[X]E[Y] is useful for mathematical derivations but is susceptible to catastrophic cancellation in numerical computation.

Did You Know?

• Covariance is defined as the expected value of the product of deviations from individual expected values.
• The sign of covariance shows the tendency in the linear relationship between variables.
• Changing the units of measurement changes the covariance value proportionally.
• The correlation coefficient normalizes covariance to a value between -1 and 1.