Calculation of the Asymptotic Covariance Matrix of the Generalized Exponential Autoregressive Model for the Huber Loss Function

Abstract

When solving regression analysis problems, the functions of least squares and least modules are most often used as a loss function. Nevertheless, they are not without drawbacks. The first function is extremely sensitive to outliers in the training data, and the second is not differentiable at zero, which complicates its use with a wide class of optimization methods. In order to offset these shortcomings, several functions have been developed that combine the capabilities of least squares and least modules estimates, for example, the Huber function. Up to a certain threshold, they behave similarly to a quadratic function; after that, they behave similarly to a modulus function. Thus, they are both differentiable across the entire definition area and insensitive to outliers (exceeding a given threshold). This paper presents the calculation of the asymptotic covariance matrix of a generalized exponential autoregressive model, the Ozaki model, for estimating the Huber function. It includes generalizing the resulting formula to a broader class of estimates, such as the least squares estimate, the least modules estimate, and other estimates combined together into a set of M-estimates

Please cite this article in English as:

Goryainov V.B., Masyagin M.M. Calculation of the asymptotic covariance matrix of the generalized exponential autoregressive model for the Huber loss function. Herald of the Bauman Moscow State Technical University, Series Natural Sciences, 2025, no. 4 (121), pp. 23--39 (in Russ.). EDN: QCRHHH

References

[1] Hastie T., Tibshirani R., Friedman Jr. The elements of statistical learning. Springer, 2009.

[2] Ozaki T. Non-linear phenomena and time series models. 43 Session of the International Statistical Institute, 1981.

[3] Hesamian Gh., Torkian F., Johannssen A., et al. An exponential autoregressive time series model for complex data. Mathematics, 2023, vol. 11, iss. 19, art. 4022. DOI: https://doi.org/10.3390/math11194022

[4] Xu H., Wan L., Ding F., et al. Fitting the exponential autoregressive model through recursive search. J. Frankl. Inst., 2019, vol. 356, iss. 11, pp. 5801--5818. DOI: https://doi.org/10.1016/j.jfranklin.2019.03.016

[5] Harvill J.L., Ray B.K. Functional coefficient autoregressive models for vector time series. Comput. Stat. Data An., 2006, vol. 50, iss. 12, pp. 3547--3566. DOI: https://doi.org/10.1016/j.csda.2005.07.016

[6] Patrick D.J., Harvill J.L., Hansen C.W. A semiparametric spatio-temporal model for solar irradiance data. Renew. Energy, 2016, vol. 87-1, pp. 15--30. DOI: https://doi.org/10.1016/j.renene.2015.10.001

[7] Kulik R., Bilayi-Biakana Cl., Soulier Ph. Statistical inference for heavy tailed series with extremal independence. Extremes, 2020, vol. 23, no. 1, pp. 1--33. DOI: https://doi.org/10.1007/s10687-019-00365-z

[8] Galvao A., Kato K. Estimation and inference for linear panel data models under misspecification when both n and T are large. J. Bus. Econ. Stat., 2014, vol. 32, iss. 2, pp. 285--309. DOI: https://doi.org/10.1080/07350015.2013.875473

[9] Sanchez C.J., Torvisco J.A.F., Rodriguez-Arias M. TAC method for fitting exponential autoregressive models and others: applications in economy and finance. Mathematics, 2021, vol. 9, iss. 8, pp. 862--882. DOI: https://doi.org/10.3390/math9080862

[10] Parker P.A., Holan S.H., Ravishanker N. Nonlinear time series classification using bispectrum-based deep convolutional neural networks. Appl. Stoch. Models Bus. Ind., 2020, vol. 36, iss. 5, pp. 877--890. DOI: https://doi.org/10.1002/asmb.2536

[11] van de Geer S.A. Empirical processes in M-estimation. Cambridge Univ. Press, 2009.

[12] Chan K.S., Tong H. On the use of the deterministic Lyapunov function for the ergodicity of stochastic difference equations. Adv. Appl. Probab., 1985, vol. 17, iss. 3, pp. 666--668. DOI: https://doi.org/10.2307/1427125

[13] Hausler E., Luschgy H. Stable convergence and stable limit theorems. Probability Theory and Stochastic Modelling, vol. 74. Cham, Springer, 2015. DOI: https://doi.org/10.1007/978-3-319-18329-9

[14] Magnus J.R., Neudecker H. Matrix differential calculus with applications in statistics and econometrics. Wiley, 2019.

[15] Goryainov V.B. Least-modules estimates for spatial autoregression coefficients. J. Comput. Syst. Sci. Int., 2011, vol. 50, no. 4, pp. 565--572. DOI: https://doi.org/10.1134/S1064230711040101

[16] Huber P.J., Ronchetti E.M. Robust statistics. Wiley, 2009.

[17] Andersen P.K., Gill R.D. Cox’s regression model for counting processes: a large sample study. Ann. Statist., 1982, vol. 10, iss. 4, pp. 1100--1120. DOI: https://doi.org/10.1214/aos/1176345976

[18] Billingsley P. Convergence of probability measures. Willey, 1999.

[19] White H. Asymptotic theory for econometricians. Academic Press, 2014.

[20] Corder G.W., Foreman D.I. Nonparametric statistics: a step-by-step approach. Willey, 2014.