Linear Correlation in Discrete mathematics

The linear correlation can be described as a measurement of dependence between two random variables.

There are various characteristics in the linear correlation, which are described as follows:

The range of linear correlation is between -1 and 1.
The linear correlation is proportional to covariance.
The interpretation of covariance and linear correlation is very similar to each other.

Definition of Linear Correlation

Suppose there are two random variables, X and Y. The linear correlation coefficient is also known as Pearson's correlation coefficient. We can define the linear correlation coefficient between the variables X and Y in the following way:

Where

Cov[X, Y] is used to indicate the covariance between X and Y.

stdev[X] stdev[Y] are used to indicate the standard deviations of X and Y.

If there is Cov[X, Y], stdev[X], and stdev[Y], only after that, we can define the linear correlation coefficient. We can often denote it as Linear Correlation in Discrete mathematics .

Zero Standard deviations

If stdev[X] and stdev[Y] both standard deviations are strictly greater than zero, only after that, the ratio will be well defined.

If one of two standard deviations is 0, only after that, we can often assume that Corr[X, Y] = 0.

When one of two standard deviations is 0, then the above assumption is also equivalent to assuming that 0/0 = 0 because of Cov[X, Y] = 0.

Interpretation

The interpretation of linear correlation and the interpretation of covariance are very similar with each. With the help of correlation between X and Y, we can see the similarities between their derivations. The range of linear correlation between -1 to 1 is described as follows:

With the help of correlation, we can easily know about the intensity of linear dependence between two random variables, which are described as follows:

The positive linear dependence between variables X and Y will be stronger if the closer correlation is 1 or it is closer to 1.
The negative linear dependence between variables X and Y will be stronger if the closer correlation is -1 or it is closer to -1.

Terminology

In the case of linear correlation, there are some terminologies that are often used. These terminologies are described as follows:

The variables X and Y will be known as positively linearly correlated if Corr[X, Y] > 0. We can simply call it positively correlated.
The variables X and Y will be known as negatively linearly correlated if Corr[X, Y] < 0. We can simply call it negatively correlated.
The variables X and Y will be known as linearly correlated if Corr[X, Y] ≠ 0. We can call it simply correlated.
The variables X and Y will be known as uncorrelated if Corr[X, Y] = 0. We can also notice that Cov[X, Y] = 0 ? Corr[X, Y] = 0. If there is a case where Cov[X, Y] = 0, then two random variables, X and Y will be known as uncorrelated.

Example: In this example, we will have two discrete random variables, and we will try to find out the coefficient of linear correlation between them.

Solution: Suppose there is a 2-dimensional random vector X, and we will indicate its entries as X1 and X2.

Now we will also assume that the support of X is:

R_X= {[-1 1], [-1 -1], [1 1]}

The joint probability mass function of X is shown below:

Linear Correlation in Discrete mathematics

The support of X₁ is described as follows:

R_X1 = {-1, 1}

The probability mass function of X₁ is shown below:

The expected value of X₁ is shown below:

The expected value of X₁² is shown below:

The variance of X₁ is shown below:

Var[X₁] = E[X₁²] - E[X₁]²

= 1 - (-1/3)² = 8/9

The standard deviation of X₁ is shown below:

Now we will explain X₂.

The support of X₂ is described as follows:

R_X2 = {-1, 1}

And probability mass function of X₂ is shown below:

The expected value of X₂ is shown below:

The expected value of X₂² is shown below:

The variance of X₂ is shown below:

Var[X₂] = E[X₂²] - E[X₂]²

= 1 - (1/3)² = 8/9

The standard deviation of X₂ is shown below:

The expected value of X₁X₂ can be computed with the help of Transformation theorem, which is described as follows:

Hence, the covariance between X₁ and X₂ is described as follows:

Cov[X₁, X₂] = E[X₁ X₂] - E[X₁] E[X₂] = 1/3 - (-1/3) * 1/3 = 4/9

The linear correlation coefficient is described as follows:

Correlation of Random variable with itself

If there is a random variable X, then it will contain the following property:

Corr[X, X] = 1

Proof:

We can prove this in the following way:

While proving this, we have used a fact that is described as follows:

Cov[X, X] = Var[X]

Symmetry

The linear correlation coefficient must be symmetric like this:

Corr[X, Y] = Corr[Y, X]

Proof:

We can prove this in the following way:

While proving this, we have use a fact covariance is symmetric, which is described as follows:

Cov[X, Y] = Cov[Y, X]

Examples of Linear correlation coefficient

There are various examples of the linear correlation coefficient, and some of them are described as follows:

Example 1:

In this example, we have a 2*1 discrete random vector which is denoted by X. The components of this vector are X₁ and X₂.

Now we will assume that the support of X is:

Its joint probability mass function is described as follows:

Here we have to calculate the coefficient of linear correlation between the components X₁ and X₂.

Solution:

The support of X₁ is described as follows:

R_X1 = {1, 2}

Its marginal probability mass function is described as follows:

The expected value of X₁ is described as follows:

The expected value of X₁² is described as follows:

The variance of X₁ is described as follows:

Var[X₁] = E[X₁²] - [X₁]²

= 8/5 - (6/5)²

= (40-36)/25

= 4/25

The standard deviation of X₁ is described as follows:

Now we will show the X₂.

So the support of X₂ is described as follows:

R_x2 = {1, 5}

Its marginal probability mass function is described as follows:

The expected value of X₂ is described as follows:

The expected value of X₂² is described as follows:

The variance of X₂ is described as follows:

Var[X₂] = E[X₂²] - [X₂]²

= 101/5 - (21/5)²

= (505-441)/25

= 64/25

The standard deviation of X₁ is described as follows:

The expected value of X₁X₂can be computed with the help of Transformation theorem like this:

Hence, the covariance between X₁ and X₂ is described as follows:

Cov[X₁, X₂] = E[X₁ X₂] - E[X₁] E[X₂]

= 22/5 - (6/5) * 21/5

= (110-126)/25

= -16/25

The coefficient of linear correlation between X₁ and X₂ is described as follows:

Example 2:

In this example, we have a 2*1 discrete random vector, which is denoted by X. The entries of X are X₁ and X₂.

Now we will assume that the support of X is:

Its joint probability mass function is described as follows:

Here we have to calculate the coefficient of linear correlation between the entries X₁ and X₂.

Solution:

The support of X₁ is described as follows:

R_X1 = {1, 2, 3}

Its marginal probability mass function is described as follows:

The mean of X₁ is described as follows:

The expected value of X₁² is described as follows:

The variance of X₁ is described as follows:

Var[X₁] = E[X₁²] - [X₁]²

= 14/3 - (2)²

= (14-12) /3

= 2/3

The standard deviation of X₁ is described as follows:

Now we will show X₂.

The support of X₂ is described as follows:

R_X2 = {1, 2, 3}

Its probability mass function is described as follows:

The mean of X₂ is described as follows:

The expected value of X₂² is described as follows:

The variance of X₂ is described as follows:

Var[X₂] = E[X₂²] - [X2]²

= 14/3 - (2)²

= (14-12) /3

= 2/3

The standard deviation of X₂ is described as follows:

The expected value of X₁X₂ can be computed with the help of Transformation theorem like this:

Hence, the covariance between X₁ and X₂ will become as follows when we put these pieces together:

Cov[X₁, X₂] = E[X₁ X₂] - E[X₁] E[X₂]

= 13/3 - 2 * 2

= (13-12) /3

= 1/3

The coefficient of linear correlation between X₁ and X₂is described as follows:

Example 3:

In this example, we have a continuous random vector [X, Y], and we will assume that the support of this vector is:

R_XY = [0, ∞) * [1, 2]

Its joint probability density function is described as follows:

Here we have to calculate the coefficient of linear correlation between X and Y.

Solution:

The support of Y is described as follows:

R_Y = [1, 2]

The marginal probability density function of Y will be zero when there is a case y ? R_Y. We can get the marginal probability density function of Y with the help of integrating x out of the joint probability density in the following way:

Thus, we can get the marginal probability density function of Y in the following way:

The expected value of Y is described as follows:

The expected value of Y² is described as follows:

The variance of Y is described as follows:

Var[Y] = E[Y²] - E[Y]²

= 7/3 - (3/2)²

= (28-27) /12

= 1/12

The standard deviation of Y is described as follows:

Now we will show X.

The support of X is described as follows:

Rx = [0, ∞)

The marginal probability density function of X will be zero when there is a case x ? R_X. We can get the marginal probability density function of X with the help of integrating y out of the joint probability density in the following way:

In this case, the integral cannot be explicitly computed for the density function but we can use the following way to write the marginal probability density function of X like this: