Instrumental weighted variables under heteroscedasticity. Part I – Consistency

Víšek, Jan Ámos

Kybernetika, Tome 53 (2017) no. 1, pp. 1-25

Abstract (VO)

The proof of consistency instrumental weighted variables, the robust version of the classical instrumental variables is given. It is proved that all solutions of the corresponding normal equations are contained, with high probability, in a ball, the radius of which can be selected - asymptotically - arbitrarily small. Then also $\sqrt{n}$-consistency is proved. An extended numerical study (the Part II of the paper) offers a picture of behavior of the estimator for finite samples under various types and levels of contamination as well as various extent of heteroscedasticity. The estimator in question is compared with two other estimators of the type of “robust instrumental variables” and the results indicate that our estimator gives comparatively good results and for some situations it is better. The discussion on a way of selecting the weights is also offered. The conclusions show the resemblance of our estimator with the $M$-estimator with Hampel's $\psi$-function. The difference is that our estimator does not need the studentization of residuals (which is not a simple task) to be scale- and regression-equivariant while the $M$-estimator does. So the paper demonstrates that we can directly compute - moreover by a quick algorithm (reliable and reasonably quick even for tens of thousands of observations) - the scale- and the regression-equivariant estimate of regression coefficients.

Détail

DOI : 10.14736/kyb-2017-1-0001

Classification : 62F35, 62J02
Keywords: weighting order statistics of the squared residuals; consistency of the instrumental weighted variables; heteroscedasticity of disturbances; numerical study

Bibliographie

[1] Amemiya, T.: Two stage least absolute deviation estimators. Econometrica 50 (1982), 689-711. | DOI | MR

[2] Atkinson, A. C., Riani, M., Cerioli, A.: Exploring Multivariate Data with the Forward Search. Springer Series in Statistics 2004, 31-88. | DOI | MR | Zbl

[3] Beran, R.: An efficient and robust adaptive estimator of location. Ann. Statist. 6 (1978), 292-313. | DOI | MR | Zbl

[4] Bowden, R. J., Turkington, D. A.: Instrumental variables. Cambridge Univ. Press, Cambridge 1984. | DOI | MR | Zbl

[5] Bramati, C. M., Croux, C.: Robust estimators for the fixed effects panel data model. The Econometr. J. 10 (2077), 521-540. | DOI | Zbl

[6] Breiman, L.: Probability. Addison-Wesley Publishing Company, London 1968. | MR | Zbl

[7] Čížek, P.: Generalized method of trimmed moments. J. Statist. Planning Inference 171 (2009), 63-78. | DOI | MR | Zbl

[8] Cochrane, D., Orcutt, G. H.: Application of least squares regression to relationhips containing autocorrelated error terms. J. Amer. Statist. Assoc. 44 (1949), 32-61. | DOI

[9] Cohen-Freue, G. V., Ortiz-Molina, H., Zamar, R. H.: A New Robust Instrumental Variables Estimator.

[10] Cohen-Freue, G. V., Ortiz-Molina, H., Zamar, R. H.: Natural robustification of the ordinary instrumental variables estimator. Biometrics 69 (2013), 641-650. | DOI | MR

[11] Croux, C., Aelst, S. Van, Dehon, C.: Bounded influence regression using high breakdown scatter matrices. Ann. Inst. Statist. Math. 55 (2013), 265-285. | DOI | MR

[12] Davies, P. L.: Asymptotic behavior of $S$-estimates of multivariate location parameters and dispersion matrices. Ann. Statist. 15 (1987), 1269-1292. | DOI | MR

[13] (1952), M. Donsker: Justification and extension of Doob's heuristic approach to the Kolmogorov-Smirnov theorems. Ann. Math. Statist. 23 (1952), 277-281. | DOI | MR

[14] Eicker, F.: Asymptotic normality and consistency of the least squares estimators for families of linear regressions. Ann. Math. Stat. 34 (1963), 447-456. | DOI | MR | Zbl

[15] Eicker, F.: Limit theorems for regression with unequal and dependent errors. In: Proc. Fifth Berkeley Symposium on Mathematical Statistics and Probability (L. Le Cam and J. Neyman, eds.), University of California Press, Berkeley 1967. | MR

[16] Fabián, Z.: Induced cores and their use in robust parametric estimation. Comm. Statist. - Theory and Methods 30 (2001), 537-555. | DOI | MR | Zbl

[17] Fabián, Z.: New measures of central tendency and variability of continuous distributions. Comm. Statist. - Theory and Methods 37 (2008), 159-174. | DOI | MR | Zbl

[18] Field, C. A., (1990), E. M. Ronchetti: Small Sample Asymptotics. Institute of Mathematical Statistics Monograph Series, Hayward 1990. | MR

[19] Fisher, R. A.: A mathematical examination of the methods of determining the accuracy of an observation by the mean error and by the mean squares error. Monthly Notes Royal Astronomical Society 80 (1920), 758-770. | DOI

[20] Fisher, R. A.: Statistical Methods for Research Workers. Second edition. (1928). | MR

[21] Galton, F.: Regression towards mediocrity in hereditary stature. J. Anthropol. Inst. 15 (1886), 246-263. | DOI

[22] Greene, W. H.: Econometric Analysis. Macmillam Press, New York 1993.

[23] Hájek, J., Šidák, Z.: Theory of Rank Test. Academic Press, New York 1967. | MR

[24] Halmos, P. R.: Applied mathematics is a bad mathematics. In: Mathematics Tomorrow (L. Steen, ed.), Springer Verlag, New York 1981, pp. 9-20. | DOI | MR

[25] Hampel, F. R., Ronchetti, E. M., Rousseeuw, P. J., Stahel, W. A.: Robust Statistics - The Approach Based on Influence Functions. J. Wiley and Sons, New York 1986. | DOI | MR

[26] Hansen, L. P.: Large sample properties of generalized method of moments estimators. Econometrica 50 (1982), 1029-1054. | DOI | MR | Zbl

[27] Harvey, A. C.: Estimating regression models with multiplicative heteroscedasticity. Econometrica 44 (1976), 461-465. | DOI | MR | Zbl

[28] Hausman, J., Newey, W., Voutersen, T., Chao, J., Swanson, N.: Instrumental variable estimation with heteroscedasticity and many instruments. Quantitative Economics 3 (2012), 211-255. | DOI | MR

[29] Helland, I. S.: Partial Least Squares Regression and Statistical Models. Scand. J. Statist. 17 (1990), 97-114. | MR | Zbl

[30] Hettmansperger, T. P., Sheather, S. J.: A cautionary note on the method of least median squares. Amer. Statist. 46 (1992), 79-83. | DOI | MR

[31] Judge, G. G., Griffiths, W. E., Hill, R. C., Lutkepohl, H., Lee, T. C.: The Theory and Practice of Econometrics. Second edition. J. Wiley and Sons, New York 1985. | MR

[32] Jurečková, J.: Regression quantiles and trimmed least squares estimator under a general design. Kybernetika 20 (1984), 345-357. | MR | Zbl

[33] Kmenta, J.: Elements of Econometrics. Macmillan Publishing Company, New York 1986. | DOI | Zbl

[34] Krasker, W. S.: Two-stage bounded-influence estimators for simultaneous equations models. J. Business Econom. Statist. 4 (1986), 437-444. | DOI

[35] Krasker, W. S., Welsch, R. E.: Resistant estimation for simultaneous - equations models using weighted instrumental variables. Econometrica 53 (1985), 1475-1488. | DOI | MR | Zbl

[36] Krishnakumar, J., Ronchetti, E.: Robust-estimators for simultaneous equations models. J. Econometr. 78 (1997), 295-314. | DOI | MR | Zbl

[37] Lopuhaa, H. P.: On the relations between $S$-estimators and $M$-estimatros of multivariate location and covariance. Ann. Statist. 17 (1989), 1662-1683. | DOI | MR

[38] Maronna, R. A., Morgenthaler, S.: Robust regression through robust covariances. Comm. Statist. - Theory and Methods 15 (1986), 1347-1365. | DOI | MR | Zbl

[39] Maronna, R. A., Yohai, V. J.: Asymptotic behaviour of general $M$-estimates for regression and scale with random carriers. Zeitschrift fűr Wahrscheinlichkeitstheorie und verwandte Gebiete 58 (1981), 7-20. | DOI | MR

[40] Maronna, R. A., Yohai, V. J.: Robust estimation in simultaneous equations models. J. Statist. Planning Inference 57 (1997), 233-244. | DOI | MR | Zbl

[41] Mašíček, L.: Optimality of the least weighted squares estimator. Kybernetika 40 (2004), 715-734. | MR | Zbl

[42] Mizon, G. E.: A simple message for autocorrelation correctors: Don't. J. Econometr. 69 (1995), 267-288. | DOI | MR | Zbl

[43] Paige, C. C., Strakoš, Z.: Scaled total least squares fundamentals. Numer. Math. 91 (2002), 117-146. | DOI | MR | Zbl

[44] Phillips, P. C. B., Solo, V.: Asymptotics for linear processes. Ann. Statist. 20 (1992), 971-1001. | DOI | MR | Zbl

[45] Popper, K. R.: The Logic of Scientific Discovery. (Logik der Forscung, Springer, Vienna 1935). Hutchinson and co., New York 1952. | MR | Zbl

[46] Portnoy, S.: Tightness of the sequence of empiric c.d.f. processes defined from regression fractiles. In: Robust and Nonlinear Time - Series Analysis (J. Franke, W. H\H{a}rdle, and D. Martin, eds.), Springer Verlag, New York 1983, pp. 231-246. | DOI | MR | Zbl

[47] Rao, R. C.: Estimation of heteroscedastic variances in linear models. J. Amer. Statist. Assoc. 65 (1970), 161-172. | DOI | MR

[48] Rao, R. C.: Linear Statistical Inference and Its Applications. J. Wiley and Sons, New York 1973. | DOI | MR | Zbl

[49] Robinson, P. M.: Asymptotically efficient estimation in the presence of heteroskedasticity of unknown form. Econometrica 55 (1987), 875-891. | DOI | MR | Zbl

[50] Ronchetti, E., Trojani, F.: Robust inference with GMM estimators. J. Econometrics 101 (2001), 37-69. | DOI | MR | Zbl

[51] Rousseeuw, P. J.: Least median of square regression. J. Amer. Statist. Assoc. 79 (1984), 871-880. | DOI | MR

[52] Rousseeuw, P. J., Leroy, A. M.: Robust Regression and Outlier Detection. J. Wiley and Sons, New York 1987. | DOI | MR | Zbl

[53] Rousseeuw, P. J., Yohai, V.: Robust regressiom by means of $S$-estimators. In: Robust and Nonlinear Time Series Analysis (J. Franke, W. Härdle, and R. D. Martin, eds.), Lecture Notes in Statistics 26 Springer Verlag, New York 1984, pp. 256-272. | DOI | MR

[54] Štěpán, J.: Teorie pravděpodobnosti (Probability Theory). Academia, Praha 1987.

[55] Huffel, S. Van: Total least squares and error-in-variables modelling: Bridging the gap between statistics, computational mathematics and enginnering. In: Proc. Computational Statistics, COMPSTAT 2004 (J. Antoch, ed.), Physica Verlag/Springer, Heidelberg 2004, pp. 539-555. | DOI | MR

[56] Víšek, J. Á.: A cautionary note on the method of Least Median of Squares reconsidered. In: Trans. Twelfth Prague Conference on Information Theory, Statistical Decision Functions and Random Processes (P. Lachout, ed.), Academy of Sciences of the Czech Republic, Praha 1994, pp. 254-259.

[57] Víšek, J. Á.: Robust instruments. In: Robust'98 (J. Antoch and G. Dohnal, eds.), Union of Czech Mathematicians and Physicists, Matfyzpress, Praha 1998, pp. 195-224.

[58] Víšek, J. Á.: The robust regression and the experiences from its application on estimation of parameters in a dual economy. In: Proc. Conference Macromodels'99, Wroclaw University 1999, pp. 424-445.

[59] Víšek, J. Á.: Regression with high breakdown point. In: Robust 2000 (J. Antoch and G. Dohnal, eds.), Union of Czech Mathematicians and Physicists, Matfyzpress, Praha 2000, pp. 324-356.

[60] Víšek, J. Á.: The least weighted squares I. The asymptotic linearity of normal equations. Bull. Czech Econometr. Soc. 9 (2002), 31-58.

[61] Víšek, J. Á.: The least weighted squares II. Consistency and asymptotic normality. Bull. Czech Econometr. Soc. 9 (2002), 1-28. | MR

[62] Víšek, J. Á.: Development of the Czech export in nineties. In: Konsolidace vládnutí a podnikání v České republice a v Evropské unii I. Umění vládnout, ekonomika, politika, Matfyzpress, Praha 2003, pp. 193-220.

[63] Víšek, J. Á.: Robustifying instrumental variables. In: Proc. COMPSTAT'2004 (J. Antoch, ed.), Physica Verlag/Springer, pp. 1947-1954. | DOI | MR

[64] Víšek, J. Á.: Instrumental weighted variables - algorithm. In: Proc. COMPSTAT 2006 (A. Rizzi and M. Vichi, eds.), Physica Verlag/Springer, Heidelberg 2006, pp. 777-786. | DOI | MR

[65] Víšek, J. Á.: Kolmogorov-Smirnov statistics in multiple regression. In: Proc. ROBUST 2006 (J. Antoch and G. Dohnal, eds.), pp. 367-374.

[66] Víšek, J. Á.: Consistency of the instrumental weighted variables. Ann. Inst. Statist. Math. 61 (2009), 543-578. | DOI | MR | Zbl

[67] Víšek, J. Á.: Robust error - term - scale estimate. In: IMS Collections. Nonparametrics and Robustness in Modern Statistical Inference and Time Series Analysis: Festschrift for Jana Jurečková, 2010, pp. 254-267. | DOI | MR

[68] Víšek, J. Á.: Heteroscedasticity resistant robust covariance matrix estimator. Bull. Czech Econometric Society 17 (2010), 33-49.

[69] Víšek, J. Á.: Consistency of the least weighted squares under heteroscedasticity. Kybernetika 47 (2011), 179-206. | MR | Zbl

[70] Víšek, J. Á.: Weak $\sqrt{n}$ - consistency of the least weighted squares under heteroscedasticity. Acta Universitatis Carolinae, Mathematica et Physica 2 (2011), 51, 71-82. | MR | Zbl

[71] Víšek, J. Á.: Empirical distribution function under heteroscedasticity. Statistics 45 (2011), 497-508. | DOI | MR | Zbl

[72] Víšek, J. Á.: Robustifying estimation of the model with fixed and random effects. Part I - Theoretical considerations. Part II - Numerical study. Workshop on Algorithm for Outliers/regressors Selection organized by Bent Nielsen, Nuffield College, Oxford 2013. Methodology and Computing in Applied Probability 17 (2014), 4, 999-1014. | DOI

[73] Wagenvoort, R., Waldmann, R.: On $B$-robust instrumental variable estimation of the linear model with panel data. J. Econometr. 106 (2002), 297-324. | DOI | MR | Zbl

[74] White, H.: A heteroskedasticity - consistent covariance matrix estimator and a direct test for heteroscedasticity. Econometrica 48 (1980), 817-838. | DOI | MR

[75] Wooldridge, J. M.: Econometric Analysis of Cross Section and Panel Data. MIT Press, Cambridge 2001. (Second edition 2008.) | MR | Zbl

[76] Wooldridge, J. M.: Introductory Econometrics. A Modern Approach. MIT Press, Cambridge 2006. (Second edition 2009.)