Properties of Covariance

If X and Y are two random variables then the covariance between them is defined as,

Cov(X,Y) = E[(X-E[X])(Y-E[Y])].

Cov(X,Y) = E[XY-E[X]Y-YE[Y]+E[X]E[Y]] = E[XY]-E[X]E[Y]-E[Y]E[Y]+E[X]E[Y] = E[XY]-E[X]E[Y].

Therefore, Cov(X,Y) = E[XY]-E[X]E[Y].

The covariance between the two variables X and Y measures the extent of the linear relationship between the two variables. Therefore, calculating the covariance can help us understand if the variables are related or not. We now list some important properties of covariance.

Properties of Covariance:

• Unlike the variance which is always positive, the covariance can be positive, negative, or zero.
• If the two variables X and Y are independent then E[XY]=E[X]E[Y], which implies that the covariance between the two variables is zero. Note that the converse is not true. If the covariance between two variables is zero we cannot conclude that the two variables are independent. We can only conclude that no linear relationship exists between the two variables.
• The covariance is unaffected by a change of origin, that is, Cov(X+a, Y+b) = Cov(X, Y) for any constants a and b.
• The covariance is affected by a change in scale as follows, Cov(cX,dY) = cd Cov(X, Y) for any constants c and d.
• We can combine the above two properties as follows, Cov(cX+a,dY+b) = cd Cov(X, Y) for any constants a, b, c, and d.
• The covariance is a bilinear function in the following sense. If X, Y, Z, and W are random variables then, Cov(X+Z, Y) = Cov(X, Y) +Cov(Z, Y). Similarly, Cov(X, Y+W) = Cov(X, Y) +Cov(X,W).
• The covariance plays an important role in the formula for the variance of sums of random variables. If X and Y are two random variables then the variance of aX+bY is given by the formula, Var(aX+bY)=a²Var(X)+b²Var(Y)+2abCov(X, Y). Hence if the two variables are independent then Cov(X, Y)=0 and we see that the variance is additive. Var(X+Y)=Var(X)+Var(Y) if X and Y are independent.
• The covariance is symmetric in the two variables, that is, Cov(X, Y) = Cov(Y, X).
• Taking the covariance of a variable with itself gives us the variance. Cov(X,X) = Var(X).
• The covariance of any random variable with a constant is always zero. Cov(X,k) = 0 for any constant k.
• Unlike the correlation coefficient that is always bounded between -1 and +1, the covariance is unbounded. It can take any value between minus infinity to plus infinity.