Read to Lead

Statistics : Correlation and Regression Tutorials

Covariance between X and Y

  • STATISTICS
Covariance between X and Y Let $(x_i, y_i), i=1,2, \cdots , n$ be $n$ pairs of observations. Covariance measures the simultaneous variability between the two variables. It indicates how the two variables are related. A positive value of covariance indicate that the two variables moves in the same direction, whereas a negative value of covariance indicate that the two variables moves on opposite direction. Formula The sample covariance between $x$ and $y$ is denoted by $Cov(x,y)$ or $s_{xy}$ and is defined as
3 min Read 2

Karl Pearson Correlation Coefficient

  • STATISTICS
Karl Pearson’s Correlation Coefficient Let $(x_i, y_i), i=1,2, \cdots , n$ be $n$ pairs of observations then the Karl Pearson’s coefficient of correlation between two variables $X$ and $Y$ is denoted by $r_{xy}$ or $r$ and is given by $r = \dfrac{Cov(X,Y)}{\sqrt{Var(X) Var(Y)}}$ where, the sample covariance between $x$ and $y$ is $$ \begin{aligned} Cov(x,y) =s_{xy}&=\frac{1}{n-1}\sum_{i=1}^{n}(x_i -\overline{x})(y_i-\overline{y})\\ &= \frac{1}{n-1}\bigg(\sum_{i=1}^n x_iy_i - \frac{(\sum_{i=1}^n x_i)(\sum_{i=1}^n y_i)}{n}\bigg) \end{aligned} $$ the sample variance of $x$ is $$ \begin{aligned} V(x) =s_{x}^2 &=\frac{1}{n-1}\sum_{i=1}^{n}(x_i -\overline{x})^2\\ &= \frac{1}{n-1}\bigg(\sum_{i=1}^n x_i^2 - \frac{(\sum_{i=1}^n x_i)^2}{n}\bigg) \end{aligned} $$
4 min Read 3

Simple linear regression from raw data

  • STATISTICS
Simple linear regression from raw data Let $(x_i, y_i), i=1,2, \cdots , n$ be $n$ pairs of observations. The simple linear regression model of $Y$ on $X$ is $$y_i=\beta_0 + \beta_1x_i +e_i$$ where, $y$ is a dependent variable, $x$ is an independent variable, $\beta_0$ is an intercept, $\beta_1$ is the slope, $e$ is the error term. Formula By the method of least square, the model parameters $\beta_0$ and $\beta_1$ can be estimated as
3 min Read 2

Simple linear regression from sum and sumsquares

  • STATISTICS
Simple linear regression from sum and sum of squares Let $(x_i, y_i), i=1,2, \cdots , n$ be $n$ pairs of observations. The simple linear regression model of $Y$ on $X$ is $$y_i=\beta_0 + \beta_1x_i +e_i$$ where, $y$ is a dependent variable, $x$ is an independent variable, $\beta_0$ is an intercept, $\beta_1$ is the slope, $e$ is the error term. Formula By the method of least square, the model parameters $\beta_0$ and $\beta_1$ can be estimated as
2 min Read 1

Simple linear regression from summarize data

  • STATISTICS
Simple linear regression from summarize data Let $(x_i, y_i), i=1,2, \cdots , n$ be $n$ pairs of observations. The simple linear regression model of $Y$ on $X$ is $$y_i=\beta_0 + \beta_1x_i +e_i$$ where, $y$ is a dependent variable, $x$ is an independent variable, $\beta_0$ is an intercept, $\beta_1$ is the slope, $e$ is the error term. Formula The simple linear regression model parameters $\beta_0$ and $\beta_1$ can be estimated using the method of least square.
2 min Read 1

Spearman Rank Correlation Coefficient

  • STATISTICS
Spearman’s Rank correlation Coefficient Let $(x_1, y_1), (x_2, y_2), \cdots , (x_n, y_n)$ be the ranks of $n$ individuals in two characteristics $A$ and $B$ respectively. Formula Then the Spearman’s rank correlation coefficient is denoted by $\varrho$ and is given by $\varrho = 1- \dfrac{6 \sum_{i=1}^{n}d_i^2}{n(n^2-1)}$ where, $d_i = x_i - y_i$ is the difference between the pairs of ranks of the $i^{th}$ individual in the two characteristics and $n$ is the number of pairs.
2 min Read 1

Testing Correlation Coefficient

  • STATISTICS
Testing Correlation Coefficient Let $(x_i, y_i), i=1,2,\cdots, n$ be a random sample of $n$ pairs of observations drawn from a bivariate normal population with correlation coeffficient $\rho$. Let $r_{xy}$ or $r$ be the observed correlation coefficient between $x$ and $y$. A test of significance for a linear relationship between the variables $x$ and $y$ can be performed using the sample correlation coefficient $r_{xy}$. We wish to test the hypothesis $H_0 : \rho =0$ (there is no significant linear relationship between $x$ and $y$) against $H_a : \rho \neq 0$ (there is a significant linear relationship between $x$ and $y$).
2 min Read 1

Testing Correlation Coefficient Examples

  • STATISTICS
Testing Significance of Linear Relationship A test of significance for a linear relationship between the variables $x$ and $y$ can be performed using the sample correlation coefficient $r_{xy}$. Example 1 For a sample of eight bears, researchers measured the distances around the bears’ chests and weighed the bears. The Sample correlation coefficient between the chest size and weight of bears is $r=0.744$ for $n=8$ bears. Using $\alpha=0.05$, determine if there is a positive linear correlation between chest size and weight.
7 min Read 5

Testing Correlation Coefficient Part 2

  • STATISTICS
Testing Correlation Coefficient Let $(X_i, Y_i), i=1,2,\cdots, n$ be a random sample of $n$ pairs of observations drawn from a bivariate population with correlation coeffficient $\rho$. Let $r$ be the observed correlation coefficient between $X$ and $Y$. We wish to test the hypothesis $H_0 : \rho =\rho_0$ against $H_a : \rho \neq \rho_0$. Assumptions The population from which, the samples drawn, is a bivariate normal. The relationship between $X$ and $Y$ is linear.
2 min Read 1

Testing Correlation Coefficient Part 2 examples

  • STATISTICS
Testing Correlation Coefficient In this tutorial we will discuss step by step solution of numerical problems on testing whether the population correlation coefficient is $\rho_0$ or not. Example 1 The median records shows that the correlation between the age of the mother and the birth weight of their first child is less than -0.34. A random sample of 8 mother’s age and the birth weight of their first child are as follows:
5 min Read 3

Testing homogeneity of Two Correlation Coefficient

  • STATISTICS
Testing Homogeneity of Two Correlation Coefficient We wish to test the null hypothesis that the correlation $\rho_1$ between $X$ and $Y$ in one population is the same as the correlation $\rho_2$ between $X$ and $Y$ in another population. Let $n_1$ be the sample pairs of observation from first population with sample correlation coefficient $r_1$ and $n_2$ be the sample pairs of observation from second population with sample correlation coefficient $r_2$.
2 min Read 1

Testing homogeneity of Two Correlation Coefficient Examples

  • STATISTICS
Testing Homogeneity of Two Correlation Coefficient In this tutorial we will discuss step by step solution to some problems on testing homogeneity of two correlation coefficients. Example 1 The correlation between the mathematics scores and statistics scores of group of 50 male students is 0.358 and the correlation between the mathematics scores and statistics scores of groups of 40 female students is 0.569. Is the relationship between mathematics scores and statistics scores for male students is weaker than for female students?
4 min Read 3