The fundamental aim of regression analysis is to determine a regression equation (line) that makes sense and fits the representative data such that the error of variance is as small as possible.
Let us consider the line of regression of y on x, that is, y = a + bx The coefficient ‘b’ which is the slope of the line of regression of y on x is called the coefficient of regression of y on x. It represents the increment in the value of the dependent variable y for a unit change in the value of the independent variable x. In other words, it represents the rate of change of y with respect to x. For notational convenience, the slope b, i.e., coefficient of regression of y on x is written as b_{yx}.
Similarly in the regression equation of x on y, that is,x = a + by, the coefficient b represents the change in the value of dependent variable x for a unit change in the value of independent variable y and is called the coefficient of regression of x on y. For notational convenience, it is written as b_{xy}. We now list out some of the properties of these regression coefficients.
Properties of Regression Coefficients:
- The correlation coefficient is the geometric mean of two regression coefficients, that is, r = \sqrt{b_{yx} × b_{xy}}
- If one regression coefficient is greater than one, then the other regression coefficient must be less than one, because the value of correlation coefficient r cannot exceed one. However, both the regression coefficients may be less than one.
- Both regression coefficients must have the same sign (either positive or negative). This property rules out the case that the two regression coefficients have opposite signs.
- The correlation coefficient will have the same sign (either positive or negative) as that of the two regression coefficients.
- The arithmetic mean of regression coefficients b_{xy} and b_{yx} is more than or equal to the correlation coefficient r, that is, \frac{b_{yx} + b_{xy}}{2}\geq r.
- Regression coefficients are independent of origin but not of scale.
- The correlation coefficient between two variables x and y is a symmetrical function between x and y, that is, r_{xy}=r_{yx}. However, the regression coefficients are not symmetric functions of x and y, that is, b_{xy}\neq b_{yx}.
References:
Fundamentals of Business Statistics – JK Sharma
