Model Averaging with Ridge Regularization
Job Market Paper
Abstract
Model averaging is an increasingly popular alternative to model selection. Ridge regression and model averaging serve a similar purpose, minimization of a mean squared error through shrinkage, though in different ways. In this paper, I propose the ridge-regularized modifications of Mallows model averaging (Hansen, 2007, Econometrica, 75) and heteroskedasticity-robust Mallows model averaging (Liu & Okui, 2013, The Econometrics Journal, 16) to utilize the capabilities of averaging and ridge regularization simultaneously. Via a simulation study, I examine the finite-sample improvements obtained by replacing least-squares with a ridge regression. Ridge-based model averaging is especially useful when one deals with sets of moderately to highly correlated predictors because the underlying ridge regression accommodates correlated predictors without blowing up estimation variance. A toy theoretical example shows that the relative reduction of the mean squared error is increasing with the strength of a correlation. I also demonstrate the superiority of the ridge-regularized modifications via empirical examples, focused on wages and economic growth.
Elastic-Net for Instrumental Variables Regression
Under revision
Abstract
nstrumental variables (IV) are commonly applied for identification of treatment effects and
subsequent policy evaluation. The use of many informative instruments improves the estimation accuracy. However, dealing with high-dimensional sets of instrumental variables of
unknown strength may be complicated and requires model selection or regularization of the
first stage regression. Currently, lasso is established as one of the most popular regularization techniques relying on the assumption of approximate sparsity. I investigate the relative
performance of the lasso and elastic-net estimators for fitting the first-stage as part of IV
estimation. As elastic-net includes a ridge-type penalty in addition to a lasso-type penalty,
it generally improves upon lasso in finite samples when correlations among the instrumental
variables are not negligible. I show that IV estimators based on the lasso and elastic-net firststage estimates can be asymptotically equivalent. Via a Monte Carlo study I demonstrate
the robustness of the sample-split elastic-net IV estimator to deviations from approximate
sparsity, and to correlation among possibly high-dimensional instruments. Finally, I provide
an empirical example that demonstrates potential improvement in estimation accuracy gained
by the use of IV estimators based on elastic-net.
Many Instruments: Implementation in Stata
with Stanislav Anatolyev, Stata Journal, 2019
Abstract
In recent decades, econometric tools for handling instrumental-variable regressions characterized by many instruments have been developed. We introduce a command, mivreg, that implements consistent estimation and testing in linear instrumental-variables regressions with many (possibly weak) instruments. mivreg covers both homoskedastic and heteroskedastic environments, estimators that are both nonrobust and robust to error nonnormality and projection matrix limit, and parameter tests and specification tests both with and without correction for existence of moments. We also run a small simulation experiment using mivreg and illustrate how mivreg works with real data.
Citation: Anatolyev, Stanislav and Alena Skolkova (2019) "Many instruments: Implementation in Stata", Stata Journal, vol. 19, no. 4, pp. 849-866