Regression Analysis : Meaning, Nature, Scope, Importance, Types, Methods, Coefficients, Lines, Equation, Properties
In this article we will discuss about regression Analysis Meaning, Nature, Scope, Importance, Types, Methods, Regression Coefficients, Regression Lines & Equations, Regression Equation & Properties of Regression Coefficients.
Meaning, Definition and Nature of Regression Analysis
Meaning of Regression Analysis
Meaning of regression analysis – A study of measuring the relationship between associated variables, wherein one variable is dependent on another independent variable, called as Regression. It is developed by Sir Francis Galton in 1877 to measure the relationship of height between parents and their children. Meaning & Nature Regression Analysis
Regression analysis is a statistical tool to study the nature and extent of functional relationship between two or more variables and to estimate (or predict) the unknown values of dependent variable from the known values of independent variable.
The variable that forms the basis for predicting another variable is known as the Independent Variable and the variable that is predicted is known as dependent variable. For example, if we know that two variables price (X) and demand (Y) are closely related we can find out the most probable value of X for a given value of Y or the most probable value of Y for a given value of X. Similarly, if we know that the amount of tax and the rise in the price of a commodity are closely related, we can find out the expected price for a certain amount of tax levy. Meaning & Nature Regression Analysis
regression Analysis Meaning, Nature, Scope, Importance, Types, Methods, Regression Coefficients, Regression Lines Lines, Regression Equation & Properties of Regression Coefficients.
Definition
Regression analysis is the measure of the average relationship between two or more variables in terms of the original units of the data.
Uses of Regression Analysis
following is the nature / importance/ uses of regression analysis
Uses of Regression Analysis:
1. It provides estimates of values of the dependent variables from values of independent variables.
2. It is used to obtain a measure of the error involved in using the regression line as a basis for estimation.
3. With the help of regression analysis, we can obtain a measure of degree of association or correlation that exists between the two variables.
4. It is highly valuable tool in economies and business research, since most of the problems of the economic analysis are based on cause and effect relationship.
Nature of Regression Analysis
Following are the nature of regression analysis – Meaning & Nature Regression Analysis
Meaning and Nature of Regression analysis is a statistical method that is used to study the relationship between a dependent variable and one or more independent variables. The nature of regression analysis can be described as follows:
- Quantitative Analysis: Regression analysis involves the use of quantitative data. The dependent variable and independent variable(s) are measured using numerical values.
- Predictive Analysis: Regression analysis is often used to make predictions about the relationship between variables. It can be used to predict the value of the dependent variable based on the value of the independent variable(s).
- Statistical Analysis: Regression analysis involves the use of statistical techniques to estimate the relationship between variables. It is based on mathematical models that use the data to estimate the parameters of the model.
- Correlative Analysis: Regression analysis is used to study the correlation between variables. It can help identify whether there is a positive or negative relationship between the variables and the strength of that relationship.
- Causal Analysis: Regression analysis can be used to study the causal relationship between variables. It can help identify whether the independent variable(s) cause changes in the dependent variable.
Overall, regression analysis is a powerful statistical tool that can help us understand the relationship between variables and make predictions based on the data. It can be used for both correlative and causal analysis, and it is based on mathematical models that use statistical techniques to estimate the relationship between variables.
regression Analysis Meaning, Nature, Scope, Importance, Types, Methods, Regression Coefficients, Regression Lines Lines, Regression Equation & Properties of Regression Coefficients.
Scope of Regression Analysis
Scope of Regression analysis is a statistical method used to examine the relationship between a dependent variable and one or more independent variables. It has a wide range of applications in various fields, including:
- Economics: Regression analysis is widely used in economics to study the relationship between various economic variables, such as GDP, inflation, unemployment, and interest rates.
- Business: Regression analysis is used in business to understand the relationship between sales and various marketing variables, such as advertising, promotion, and pricing.
- Healthcare: Regression analysis is used in healthcare to study the relationship between various health factors, such as diet, exercise, smoking, and medical conditions.
- Social Sciences: Regression analysis is used in social sciences to examine the relationship between various social factors, such as income, education, race, and gender.
- Environmental Science: Regression analysis is used in environmental science to study the relationship between environmental factors, such as pollution, climate change, and natural disasters.
- Engineering: Regression analysis is used in engineering to study the relationship between various engineering variables, such as temperature, pressure, and flow rate.
Overall, regression analysis is a powerful statistical tool that can be applied to various fields to understand the relationship between variables and make predictions based on the data. Meaning & Nature Regression Analysis
regression Analysis Meaning, Nature, Scope, Importance, Types, Methods, Regression Coefficients, Regression Lines Lines, Regression Equation & Properties of Regression Coefficients.
Types of Regression Analysis
There are several types of regression analysis, each with its own specific purpose and assumptions. Here are some of the most common types:
- Simple Linear Regression: Simple linear regression is used when there is a single independent variable that is used to predict the value of a dependent variable. It assumes that the relationship between the variables is linear.
- Multiple Linear Regression: Multiple linear regression is used when there are two or more independent variables that are used to predict the value of a dependent variable. It assumes that the relationship between the variables is linear.
- Polynomial Regression: Polynomial regression is used when the relationship between the independent and dependent variables is nonlinear. It uses polynomial functions to model the relationship between the variables.
- Logistic Regression: Logistic regression is used when the dependent variable is categorical, such as a binary outcome (yes/no) or a multi-category outcome (low/medium/high). It models the relationship between the independent variables and the probability of the outcome.
- Ridge Regression: Ridge regression is used when there is multicollinearity (high correlation) between the independent variables in multiple linear regression. It adds a penalty term to the regression model to reduce the impact of multicollinearity.
- Time Series Regression: Time series regression is used when the data is collected over time and there is a relationship between the dependent variable and time. It models the relationship between the dependent variable and time, as well as any other independent variables that may affect the outcome.
Overall, the choice of regression analysis depends on the nature of the data and the research question. Each type of regression analysis has its own assumptions and requirements, and it is important to choose the appropriate method to ensure accurate results.
regression Analysis Meaning, Nature, Scope, Importance, Types, Methods, Regression Coefficients, Regression Lines Lines, Regression Equation & Properties of Regression Coefficients.
Following are the Regression Lines and Equations
Linear Regression Lines and Equation
Regression Equations
If two variables have linear relationship then as the independent variable (X) changes, the dependent variable (Y) also changes. If the different values of X and Y are plotted, then the two straight lines of best fit can be made to pass through the plotted points. These two lines are known as regression lines. Again, these regression lines are based on two equations known as regression equations. These equations show best estimate of one variable for the known value of the other. The equations are linear.
Linear regression equation of
Y on X is Y = a + bX ……. (1)
And X on Y is X = a + bY……. (2)
a, b are constants.
From (1) We can estimate Y for known value of X.
(2) We can estimate X for known value of Y
If the values of constants “a” and “b” are obtained, the line is completely determined. But the question is how to obtain these values. The answer is provided by the method of least squares. With the little algebra and differential calculus, it can be shown that the following two normal equations, if solved simultaneously, will yield the values of the parameters “a” and “b”.
Two normal equations:
This above method is popularly known as direct method, which becomes quite cumbersome when the values of X and Y are large. This work can be simplified if instead of dealing with actual values of X and Y, we take the deviations of X and Y series from their respective means. In that case:
Regression equation Y on X:
| Y = a + bX | will change to | (Y – Ẏ) = byx (X – Ẋ) |
| Regression equation X on Y: | ||
| X = a + bY | will change to | (X – Ẋ) = bxy (Y – Ẏ) |
In this new form of regression equation, we need to compute only one parameter i.e. “b”. This “b” which is also denoted either “byx” or “bxy” which is called as regression coefficient.
regression Analysis Meaning, Nature, Scope, Importance, Types, Methods, Regression Coefficients, Regression Lines Lines, Regression Equation & Properties of Regression Coefficients.
Regression Lines
After regression equations, there are two regression lines X on Y and Y on X
For regression analysis of two variables there are two regression lines, namely Y on X and X on Y. The two regression lines show the average relationship between the two variables. For perfect correlation, positive or negative i.e., r = + 1, the two lines coincide i.e., we will find only one straight line. If r = 0, i.e., both the variables are independent then the two lines will cut each other at right angle. In this case the two lines will be parallel to X and Y-axes.
Lastly the two lines intersect at the point of means of X and From this point of intersection, if a straight line is drawn on X- axis, it will touch at the mean value of x. Similarly, a perpendicular drawn from the point of intersection of two regression lines on Y- axis will touch the mean value of Y.
Therefore, with the help of simple linear regression model we have the following two regression lines
1. Regression line of Y on X: This line gives the probable value of Y (Dependent variable) for any given value of X (Independent variable).
| Regression line of Y on X OR | : : | Y – Ẏ = byx (X – Ẋ) OR
Y = a + bX |
2. Regression line of X on Y: This line gives the probable value of X (Dependent variable) for any given value of Y (Independent variable).
| Regression line of X on Y OR | : : | X – Ẋ = bxy (Y – Ẏ) OR
X = a + bY |
In the above two regression lines or regression equations, there are two regression parameters, which are “a” and “b”. Here “a” is unknown constant and “b” which is also denoted as “byx” or “bxy”, is also another unknown constant popularly called as regression coefficient. Hence, these “a” and “b” are two unknown constants (fixed numerical values) which determine the position of the line completely. If the value of either or both of them is changed, another line is determined. The parameter “a” determines the level of the fitted line (i.e. the distance of the line directly above or below the origin). The parameter “b” determines the slope of the line (i.e. the change in Y for unit change in X) Regression Lines & Equations
regression Analysis Meaning, Nature, Scope, Importance, Types, Methods, Regression Coefficients, Regression Lines Lines, Regression Equation & Properties of Regression Coefficients.
Regression Coefficients
The quantity “b” in the regression equation is called as the regression coefficient or slope coefficient. Since there are two regression equations, therefore, we have two regression coefficients.
1. Regression Coefficient X on Y, symbolically written as “bxy”
2. Regression Coefficient Y on X, symbolically written as “byx” Different formula’s used to compute regression coefficients:
Methods of Regression Analysis
The various methods can be represented in the form of chart given below:
1. Graphic Method:
Scatter Diagram:
Under this method the points are plotted on a graph paper representing various parts of values of the concerned variables. These points give a picture of a scatter diagram with several points spread over. A regression line may be drawn in between these points either by free hand or by a scale rule in such a way that the squares of the vertical or the horizontal distances (as the case may be) between the points and the line of regression so drawn is the least. In other words, it should be drawn faithfully as the line of best fit leaving equal number of points on both sides in such a manner that the sum of the squares of the distances is the best.
2. Algebraic Methods:
i. Regression Equation
The two regression equations for X on Y; X = a + bY
And for Y on X; Y = a + bX
Where X, Y are variables, and a,b are constants whose values are to be determined
For the equation, X = a + bY The normal equations are
From these normal equations the values of a and b can be determined.
Example 1:
Find the two regression equations from the following data:
| X: | 6 | 2 | 10 | 4 | 8 |
| Y: | 9 | 11 | 5 | 8 | 7 |
Solution:
| X | Y | X2 | Y2 | XY |
| 6 | 9 | 36 | 81 | 54 |
| 2 | 11 | 4 | 121 | 22 |
| 10 | 5 | 100 | 25 | 50 |
| 4 | 8 | 16 | 64 | 32 |
| 8 | 7 | 64 | 49 | 56 |
| 30 | 40 | 220 | 340 | 214 |
Regression equation of Y on X is Y = a + bX and the normal equations are
Example-2:
regression Analysis Meaning, Nature, Scope, Importance, Types, Methods, Regression Coefficients, Regression Lines Lines, Regression Equation & Properties of Regression Coefficients.
ii. Regression Coefficient
The quantity “b” in the regression equation is called as the regression coefficient or slope coefficient. Since there are two regression equations, therefore, we have two regression coefficients.
1. Regression Coefficient X on Y, symbolically written as “bxy”
2. Regression Coefficient Y on X, symbolically written as “byx” Different formula’s used to compute regression coefficients:
Example-3:
Example-4:
regression Analysis Meaning, Nature, Scope, Importance, Types, Methods, Regression Coefficients, Regression Lines Lines, Regression Equation & Properties of Regression Coefficients.
Properties of Regression Coefficients
1. The coefficient of correlation is the geometric mean of the two regression coefficients. Symbolically 
2. If one of the regression coefficients is greater than unity, the other must be less than unity, since the value of the coefficient of correlation cannot exceed unity. For example if bxy = 1.2 and byx = 1.4 “r” would be = √1.2 ∗ 1.4 = 1.29, which is not possible.
3. Both the regression coefficient will have the same sign. i.e. they will be either positive or negative. In other words, it is not possible that one of the regression coefficients is having minus sign and the other plus sign.
4. The coefficient of correlation will have the same sign as that of regression coefficient, i.e. if regression coefficient have a negative sign, “r” will also have negative sign and if the regression coefficient have a positive sign, “r” would also be positive. For example, if bxy = -0.2 and byx = -0.8 then r = – √0.2 ∗ 0.8 = – 0.4
5. The average value of the two regression coefficient would be greater than the value of coefficient of correlation. In symbol (bxy + byx) / 2 > r. For example, if bxy = 0.8 and byx = 0.4 then average of the two values = (0.8 + 0.4) / 2 = 0.6 and the value of r = r = √0.8 ∗ 0.4 = 0.566 which less than 0.6
6. Regression coefficients are independent of change of origin but not scale.
This is end of regression Analysis Meaning, Nature, Scope, Importance, Types, Methods, Regression Coefficients, Regression Lines Lines, Regression Equation & Properties of Regression Coefficients.
Download Complete eBook of Business Statistics
