No menu items!

Covariance Properties – Some Properties of Covariance

-

If X and Y are two random variables then the covariance between them is defined as,

\begin{align*}Cov(X,Y) &= E[(X-E[X])(Y-E[Y])] \\ &= E[XY-E[X]Y-YE[Y]+E[X]E[Y]] \\ &= E[XY]-E[X]E[Y]-E[Y]E[Y]+E[X]E[Y] \\ &= E[XY]-E[X]E[Y]\end{align*}

In case we are dealing with a discrete set of data values the covariance can be calculated using the formula:

Cov(X,Y) = \frac{1}{n}\sum_{i=1}^{n} (x_i - \bar{x})(y_i - \bar{y})

The covariance between the two variables X and Y measures the extent of the linear relationship between the two variables. Therefore, calculating the covariance can help us understand if the variables are related or not. We now list some important properties of covariance.

Properties of Covariance:

  1. Unlike the variance which is always positive, the covariance can be positive, negative or zero.
  2. If the two variables X and Y are independent then E[XY]=E[X]E[Y], which implies that the covariance between the two variables is zero. Note that the converse is not true. If the covariance between two variables is zero we cannot conclude that the two variables are independent. We can only conclude that no linear relationship exists between the two variables.
  3. The covariance is unaffected by change of origin, that is, Cov(X+a,Y+b) = Cov(X,Y) \text{ for any constants a and b.}
  4. The covariance is affected by change in scale as follows, Cov(cX,dY) = cd Cov(X,Y) \text{ for any constants c and d.}
  5. We can combine the above two properties as follows, Cov(cX+a,dY+b) = cd Cov(X,Y) \text{ for any constants a, b, c and d.}
  6. The covariance is a bilinear function in the following sense. If X, Y, Z and W are random variables then, \begin{align} Cov(X+Z,Y) &= Cov(X,Y) +Cov(Z,Y) \\ Cov(X,Y+W) &= Cov(X,Y) +Cov(X,W) \end{align}
  7. The covariance plays an important role in the formula for variance of sums of random variables. If X and Y are two random variables then the variance of aX+bY is given by the formula, Var(aX+bY)=a^2Var(X)+b^2Var(Y)+2abCov(X,Y). Hence if the two variables are independent then Cov(X,Y)=0 and we see that the variance is additive. Var(X+Y)=Var(X)+Var(Y)\text{ if X and Y are independent.}
  8. As a consequence of the Cauchy-Schwartz inequality we have the following, |Cov(X,Y)| \leq \sqrt{Var(X)}\sqrt{Var(Y)}. As a consequence of the above inequality, we can conclude that the correlation coefficient always ranges between -1 and 1.
  9. The covariance is symmetric in the two variables, that is, Cov(X,Y) = Cov(Y,X).
  10. Taking the covariance of a variable with itself gives us the variance. Cov(X,X) = Var(X).
  11. The covariance of any random variable with a constant is always zero. Cov(X,k) = 0 \text{ for any constant }k
  12. Unlike the correlation coefficient that is always bounded between -1 and +1, the covariance is unbounded. It can take any value between minus infinity to plus infinity.

Hey 👋

I'm currently pursuing a Ph.D. in Maths. Prior to this, I completed my master's in Maths & bachelors in Statistics.

I created this website for explaining maths and statistics concepts in the simplest possible manner.

If you've found value from reading my content, feel free to support me in even the smallest way you can.



Share this article

Recent posts

Popular categories

Recent comments