Variance, Covariance, Correlation and Covariance Matrix
Variance is a special case of covariance when two random variables are the same. Correlation is standardized covariance which is unit agnostic metrics, so you could compare any two pairs of random variables without worrying about their units. Covariance and Variance Let us look into covariance first. By definition, covariance is defined as below: $Cov(X,Y)=E[(X - E(X))(Y - E(Y))]$ Using linearity of expectation, you can easily get $Cov(X,Y)=E(XY) - E(X)E(Y)$ In statistics, i always would like to use a non-trivial example to demonstrate a concept. Here, i will demonstrate one discrete case and one continuous case. 1) Let's say we have two discrete random variables X and Y. the E(X) = 2, and E(Y) = 3, so now we can easily calculate the Cov(X,Y) as below: $Cov(X,Y)= \sum f(x,y)(x-E(X))(y-E(Y))$ $= 0.15*(1-2)*(2-3) + 0*(1-2)*(3-3)+0.15*(1-2)*(4-3)$ $+0.1*(2-2)*(2-3)+0.15*(2-2)*(3-3)+0*(2-2)*(4-3)$ $+0*(3-2)*(2-3)+0.15*(3-2)*(3-3)+0.3*(3-2)*(4-3) = 0.3$ 2)...