Alignment of vector fields on manifolds via contraction mappings

O. N. Kachan ; Yu. A. Yanovich ; E. N. Abramov

Učënye zapiski Kazanskogo universiteta. Seriâ Fiziko-matematičeskie nauki, Uchenye Zapiski Kazanskogo Universiteta. Seriya Fiziko-Matematicheskie Nauki, Tome 160 (2018) no. 2, pp. 300-308

Résumé

According to the manifold hypothesis, high-dimensional data can be viewed and meaningfully represented as a lower-dimensional manifold embedded in a higher dimensional feature space. Manifold learning is a part of machine learning where an intrinsic data representation is uncovered based on the manifold hypothesis. Many manifold learning algorithms were developed. The one called Grassmann & Stiefel eigenmaps (GSE) has been considered in the paper. One of the GSE subproblems is tangent space alignment. The original solution to this problem has been formulated as a generalized eigenvalue problem. In this formulation, it is plagued with numerical instability, resulting in suboptimal solutions to the subproblem and manifold reconstruction problem in general. We have proposed an iterative algorithm to directly solve the tangent spaces alignment problem. As a result, we have obtained a significant gain in algorithm efficiency and time complexity. We have compared the performance of our method on various model data sets to show that our solution is on par with the approach to vector fields alignment formulated as an optimization on the Stiefel group.

Détail

Keywords: manifold learning, dimensionality reduction, numerical optimization, vector field estimation.

Bibliographie

[1] Giraud Ch., Introduction to High-Dimensional Statistics, Chapman and Hall/CRC, New York, 2014, 270 pp.

[2] Donoho D. L., “High-dimensional data analysis: The curses and blessings of dimensionality”, AMS Conf. on Math Challenges of 21st Century (2000), 1–31 | MR

[3] Ezuz D., Solomon J., Kim V. G., Ben-Chen M., “GWCNN: A metric alignment layer for deep shape analysis”, Comput. Graphics Forum, 36:5 (2017), 49–57 | DOI

[4] Qiu A., Lee A., Tan M., Chung M. K., “Manifold learning on brain functional networks in aging”, Med. Image Anal., 20:1 (2015), 52–60 | DOI

[5] Bronstein M. M., Bruna J., LeCun Y., Szlam A., Vandergheynst P., “Geometric deep learning: Going beyond Euclidean data”, IEEE Signal Process. Mag., 34:4 (2017), 18–42 | DOI

[6] Xu C., Govindarajan L. N., Zhang Y., Cheng L., “Lie-X: Depth image based articulated object pose estimation, tracking, and action recognition on lie groups”, Int. J. Comput. Vision, 123:3 (2017), 454–478 | DOI | MR

[7] Seung H. S., Lee D. D., “COognition. The manifold ways of perception”, Science, 290:5500 (2000), 2268–2269 | DOI

[8] Zeestraten M. J.A., Havoutis I., Silverio J., Calinon S., Caldwell D. G., “An approach for imitation learning on riemannian manifolds”, IEEE Rob. and Autom. Lett., 2:3 (2017), 1240–1247 | DOI

[9] Klingensmith M., Koval M. C., Srinivasa S. S., Pollard N. S., Kaess M., “The manifold particle filter for state estimation on high-dimensional implicit manifolds”, 2017 IEEE Int. Conf. on Robotics and Automation (ICRA) (Singapore, 2017), 4673–4680 | DOI

[10] Bush K., Pineau J., “Manifold embeddings for model-based reinforcement learning under partial observability”, Advances in Neural Information Processing Systems, eds. Bengio Y., Schuurmans D., Lafferty J. D., Williams C. K. I., Culotta A., Curran Associates, Inc., 2009, 189–197

[11] Brahma P., Wu D., She Y., “Why deep learning works: A manifold disentanglement perspective”, IEEE Trans. Neural Networks and Learn. Syst., 27:10 (2016), 1997–2008 | DOI | MR

[12] Belkin M., Niyogi P., “Laplacian Eigenmaps for dimensionality reduction and data representation”, Neural Comput., 15:6 (2003), 1373–1396 | DOI | Zbl

[13] Roweis S. T., Saul L. K., “Nonlinear dimensionality reduction by locally linear embedding”, Science, 290:5500 (2000), 2323–2326 | DOI

[14] Zhang Z., Zha H., “Principal manifolds and nonlinear dimensionality reduction via tangent space alignment”, SIAM J. Sci. Comput., 26:1 (2004), 313–338 | DOI | MR | Zbl

[15] Bernstein A., Kuleshov A. P., “Manifold learning: Generalizing ability and tangent proximity”, Int. J. Software Inf., 7:3 (2013), 359–390

[16] Bernstein A., Kuleshov A., Yanovich Y., “Manifold learning in regression tasks”, Lecture Notes in Computer Science, 9047, 2015, 414–423 | DOI

[17] Bernstein A. V., Kuleshov A. P., Yanovich Yu. A., “Locally isometric and conformal parameterization of image manifold”, Proc. 8th Int. Conf. on Machine Vision (Barcelona, 2015), Proc. SPIE, 9875, eds. Verikas A., Radeva P., Nikolaev D., 987507, 1–7 | DOI

[18] Yanovich Yu., “Asymptotic properties of local sampling on manifold”, J. Math. Stat., 12:3 (2016), 157–175 | DOI

[19] Yanovich Yu., “Asymptotic properties of nonparametric estimation on manifold”, Proc. 6th Workshop on Conformal and Probabilistic Prediction and Applications, PMLR, 60, 2017, 18–38

[20] Bernstein A., Burnaev E., Erofeev P., “Comparative study of nonlinear methods for manifold learning”, Proc. Conf. “Information Technologies and Systems”, 2012, 85–91

[21] Bishop C. M., Pattern Recognition and Machine Learning, Springer, New York, 2006, xx+738 pp. | MR | Zbl

[22] Golub G. H., van Loan Ch. F., Matrix Computations, Johns Hopkins Univ. Press, Baltimore, 1996, 694 pp. | MR | Zbl

[23] Abramov E., Yanovich Yu. Smooth vector fields estimation on manifolds by optimization on Stiefel group, Uchenye Zapiski Kazanskogo Universiteta. Seriya Fiziko-Matematicheskie Nauki, 160:2 (2018), 220–228