A Comparative Study on Bias Regression Methods in the Presence of Multicollinearity Based on Gamma and Chi Square Distributions

Abstract

The aim of this study is to compare some regression methods in the presence of multicollinearity problem. This problem makes the estimated regression coefficients by least squares method to be conditional upon the correlated predictor variables in the model. It is also a condition in a set of regression data that have two or more regressors which are redundant and have the same information. Therefore, some regression methods that handle with multicollinearity such as partial least square regression (PLSR), ridge regression (RR) and lasso regression (LR) had reported. In this paper, the methods were compared using simulated data that follows gamma and chi square distributions with P=4 and 10, and n=60 and 90. All results were compared with each other through Mean Square Log Error (MSLE), Mean Absolute Error (MAE) and R² of their estimated values for different methods. The results show that when P=4 and n=60 RR is better methods with gamma distribution, but with chi square distribution PLRS is better methods. Also, when P=4 and n=90, RR shows better results with gamma distribution but with chi square distribution all methods have equal predictive ability. However, at P=10 and n=60 RR performed better with both gamma and chi square distributions while RR shows better results at both gamma and chi square distributions when P=10 and n=90.

Keywords: Multicollinearity, Partial Least Square Regression, Ridge Regression, Principal Component Regression