Background
Ridge regression is a technique in regression analysis that is used when independent variables (explanatory variables) are highly correlated. This high correlation can cause issues in the standard Ordinary Least Squares (OLS) method, making estimates highly sensitive and unreliable.
Historical Context
Ridge regression emerged as a solution to the problem of multicollinearity that often affects regression analysis. It was primarily developed and popularized in the 1960s and 1970s as a practical alternative to OLS when dealing with multicollinear data sets.
Definitions and Concepts
Ridge regression modifies the standard regression model by adding a degree of bias to the regression estimates. This bias is introduced through a penalty term proportional to the sum of the squares of the coefficients. The main idea is to minimize:
$$ | y - Xw |^2_2 + \lambda | w |^2_2 $$
where \(y\) is the dependent variable, \(X\) is the matrix of explanatory variables, \(w\) represents the regression coefficients, and \(\lambda\) is the tuning parameter that controls the amount of shrinkage.
Key Concepts
- Multicollinearity: A scenario in which two or more explanatory variables in a regression model are highly linearly related.
- Biased Estimator: Ridge regression introduces bias to the estimator to gain lower variance, as opposed to the unbiased estimator in OLS.
Major Analytical Frameworks
Classical Economics
While classical economics doesn’t directly engage with ridge regression, the improved reliability of regression estimates due to the reduced variance of ridge regression can benefit economic modeling and predictions.
Neoclassical Economics
In neoclassical economics, accurate estimation of agents’ behavior and market outcomes often requires dealing with multicollinear variables, making ridge regression a valuable tool.
Keynesian Economic
Keynesian economics, which frequently deals with macroeconomic models incorporating interdependent variables, can also benefit from the stabilizing influence of ridge regression.
Marxian Economics
Though Marxian economists generally employ different types of critiques and analysis, ridgeregression can still be useful for empirical studies involving multivariable relationships.
Institutional Economics
Institutional economists focusing on the influence of institutions often use empirical models where ridge regression can be applied to avoid multicollinearity issues within systemic variables.
Behavioral Economics
Behavioral economists can use ridge regression for analyzing multicollinear data in experiments assessing decision-making under uncertainty.
Post-Keynesian Economics
Post-Keynesian economists deal with models that often include correlated industrial and financial variables, potentially necessitating ridge regression techniques for reliable parameter estimation.
Austrian Economics
While Austrian economics is less reliant on empirical modeling, any quantitative analysis typical of economic history or market predictions can employ ridge regression methods.
Development Economics
Development economists working with data containing multiple correlated socio-economic indicators can use ridge regression to draw more reliable conclusions.
Monetarism
Monetarism’s focus on the relationships between money supply, inflation, and national income can benefit from the stability provided by ridge regression in longitudinal studies.
Comparative Analysis
Ordinary Least Squares (OLS) estimators might be unbiased but suffer from high variance in the context of multicollinear explanatory variables. Ridge regression, while producing biased estimates, reduces the variance, enabling more reliable predictions and inferences.
Case Studies
- Economic Growth and Development: Analyzing the relationship between various growth indicators where multicollinearity is present.
- Consumer Behavior Models: Explaining purchase decisions using several correlated psychosocial factors.
Suggested Books for Further Studies
- “The Elements of Statistical Learning” by Trevor Hastie, Robert Tibshirani, and Jerome Friedman.
- “Applied Multivariate Statistical Analysis” by Richard A. Johnson and Dean W. Wichern.
- “Regression Modeling Strategies” by Frank Harrell.
Related Terms with Definitions
- Multicollinearity: When two or more explanatory variables in a regression model are highly correlated, leading to unreliable estimations in OLS.
- Ordinary Least Squares (OLS): A type of linear least squares method for estimating the unknown parameters in a linear regression model, typically yielding unbiased but high-variance estimators in the presence of multicollinearity.
- Shrinkage: The process of imposing a penalty (multiplier) on the size of the coefficients to prevent overfitting and reduce variance.