Error Guarantees for Least Squares Approximation with Noisy Samples in Domain Adaptation

Bartel, Felix¹

¹ Chemnitz University of Technology, Faculty of Mathematics, 09107 Chemnitz, Germany

The SMAI Journal of computational mathematics, Tome 9 (2023), pp. 95-120

Résumé

Given $n$ samples of a function $f:D\to ℂ$ in random points drawn with respect to a measure ${\varrho }_S$ we develop theoretical analysis of the $L_2(D,{\varrho }_T)$-approximation error. For a parituclar choice of ${\varrho }_S$ depending on ${\varrho }_T$, it is known that the weighted least squares method from finite dimensional function spaces $V_m$, $dim\left(V_m\right)=m<\infty$ has the same error as the best approximation in $V_m$ up to a multiplicative constant when given exact samples with logarithmic oversampling. If the source measure ${\varrho }_S$ and the target measure ${\varrho }_T$ differ we are in the domain adaptation setting, a subfield of transfer learning. We model the resulting deterioration of the error in our bounds.

Further, for noisy samples, our bounds describe the bias-variance trade off depending on the dimension $m$ of the approximation space $V_m$. All results hold with high probability.

For demonstration, we consider functions defined on the $d$-dimensional cube given in unifom random samples. We analyze polynomials, the half-period cosine, and a bounded orthonormal basis of the non-periodic Sobolev space $H_{\mathrm{mix}}^2$. Overcoming numerical issues of this $H_{\mathrm{mix}}^2$ basis, this gives a novel stable approximation method with quadratic error decay. Numerical experiments indicate the applicability of our results.

Détail

Publié le : 2023-07-31

DOI : 10.5802/smai-jcm.96

Classification : 41A10, 41A25, 41A60, 41A63, 42C10, 65TXX, 65F22, 65D15, 94A20
Keywords: domain adaptation, individual function approximation, least squares, sampling theory, transfer learning, unit cube, polynomial approximation