The fundamental aim of regression analysis is to determine a regression equation (line) that makes sense and fits the representative data such that the error of variance is as small as possible.
Let us consider the line of regression of y on x, that is, y = a + bx. The coefficient ‘b’ which is the slope of the line of regression of y on x is called the coefficient of regression of y on x.
It represents the increment in the value of the dependent variable y for a unit change in the value of the independent variable x. In other words, it represents the rate of change of y with respect to x.
For notational convenience, the slope b, i.e., coefficient of regression of y on x is written as byx.
Similarly in the regression equation of x on y, that is, x = a + by, the coefficient b represents the change in the value of dependent variable x for a unit change in the value of independent variable y and is called the coefficient of regression of x on y. For notational convenience, it is written as bxy.
We now list out some of the properties of these regression coefficients.
Properties of Regression Coefficients:
- The correlation coefficient is the geometric mean of two regression coefficients, that is, r2 = bxy x byx.
- If one regression coefficient is greater than one, then the other regression coefficient must be less than one, because the value of correlation coefficient r cannot exceed one. However, both regression coefficients may be less than one.
- Both regression coefficients must have the same sign (either positive or negative). This property rules out the case that the two regression coefficients have opposite signs.
- The correlation coefficient will have the same sign (either positive or negative) as that of the two regression coefficients.
- The arithmetic mean of regression coefficients bxy and byx is more than or equal to the correlation coefficient r, that is, (bxy+byx)/2 ≥ r.
- Regression coefficients are independent of change origin.
- Regression coefficients are not independent of change of scale.
- The correlation coefficient between two variables x and y is a symmetrical function between x and y, that is, rxy=ryx. However, the regression coefficients are not symmetric functions of x and y, that is, bxy ≠ byx.