TY - CHAP
T1 - Why LASSO, EN, and CLOT
T2 - Invariance-Based Explanation
AU - Alkhatib, Hamza
AU - Neumann, Ingo
AU - Kreinovich, Vladik
AU - Van Le, Chon
N1 - Funding Information:
Acknowledgements This work was supported by the Institute of Geodesy, Leibniz University of Hannover. It was also supported in part by the US National Science Foundation grants 1623190 (A Model of Change for Preparing a New Generation for Professional Practice in Computer Science) and HRD-1242122 (Cyber-ShARE Center of Excellence). This paper was written when V. Kreinovich was visiting Leibniz University of Hannover.
PY - 2020/11/14
Y1 - 2020/11/14
N2 - In many practical situations, observations and measurement results are consistent with many different models—i.e., the corresponding problem is ill-posed. In such situations, a reasonable idea is to take into account that the values of the corresponding parameters should not be too large; this idea is known as regularization. Several different regularization techniques have been proposed; empirically the most successful are LASSO method, when we bound the sum of absolute values of the parameters, and EN and CLOT methods in which this sum is combined with the sum of the squares. In this paper, we explain the empirical success of these methods by showing that they are the only ones which are invariant with respect to natural transformations—like scaling which corresponds to selecting a different measuring unit.
AB - In many practical situations, observations and measurement results are consistent with many different models—i.e., the corresponding problem is ill-posed. In such situations, a reasonable idea is to take into account that the values of the corresponding parameters should not be too large; this idea is known as regularization. Several different regularization techniques have been proposed; empirically the most successful are LASSO method, when we bound the sum of absolute values of the parameters, and EN and CLOT methods in which this sum is combined with the sum of the squares. In this paper, we explain the empirical success of these methods by showing that they are the only ones which are invariant with respect to natural transformations—like scaling which corresponds to selecting a different measuring unit.
UR - http://www.scopus.com/inward/record.url?scp=85096207492&partnerID=8YFLogxK
U2 - 10.1007/978-3-030-48853-6_2
DO - 10.1007/978-3-030-48853-6_2
M3 - Contribution to book/anthology
AN - SCOPUS:85096207492
SN - 978-3-030-48852-9
T3 - Studies in Computational Intelligence
SP - 37
EP - 50
BT - Data Science for Financial Econometrics
PB - Springer Science and Business Media Deutschland GmbH
CY - Cham
ER -