Neural network approximation of several variable functions

D. V. Alexeev

Fundamentalʹnaâ i prikladnaâ matematika, Tome 15 (2009) no. 3, pp. 9-21.

Résumé

The main result of the work is as follows: in the Chebyshev–Hermite weighted integral metric, it is possible to approximate any function of sufficiently general form by neural network. The approximating net consists of two layers, where the first uses any predefined sigmoid function of activation and second uses linear-threshold function. The Chebyshev–Hermit weight is chosen because it lets one to imitate receptors distribution for example in an eye of a human or some mammal.

Bibliographie

[1] Arnold V. I., “O predstavlenii lyuboi nepreryvnoi funktsii trëkh peremennykh v vide summy funktsii ne bolee dvukh peremennykh”, DAN SSSR, 114:4 (1957), 679–681 | MR | Zbl

[2] Arnold V. I., “O predstavlenii funktsii neskolkikh peremennykh v vide superpozitsii funktsii menshego chisla peremennykh”, Mat. prosveschenie, 3 (1958), 41–61 | Zbl

[3] Babin D. N., “O polnote dvukhmestnykh ogranichenno determinirovannykh funktsii otnositelno superpozitsii”, Diskret. mat., 1:4 (1989), 86–91 | MR | Zbl

[4] Bernshtein S. N., “O nailuchshem priblizhenii nepreryvnykh funktsii posredstvom mnogochlenov dannoi stepeni”, Sobr. soch., T. 2, Izd-vo AN SSSR, M., 1952, 11–104

[5] Vitushkin A. G., Khenkin G. M., “Lineinye superpozitsii funktsii”, Uspekhi mat. nauk, 22:1 (1967), 77–124 | MR | Zbl

[6] Kolmogorov A. N., “O predstavlenii nepreryvnykh funktsii neskolkikh peremennykh superpozitsiyami nepreryvnykh funktsii menshego chisla peremennykh”, DAN SSSR, 108 (1956), 179–182 | MR | Zbl

[7] Kolmogorov A. N., “O predstavlenii nepreryvnykh funktsii neskolkikh peremennykh v vide superpozitsii nepreryvnykh funktsii odnogo peremennogo i slozheniya”, DAN SSSR, 114 (1957), 953–956 | MR | Zbl

[8] Mak-Kallok U., Pitts U., “Logicheskoe ischislenie idei, otnosyaschikhsya k nervnoi aktivnosti”, Avtomaty, Izd. inostr. lit., M., 1956, 362–384

[9] Marchenkov S. S., “Ob odnom metode analiza superpozitsii nepreryvnykh funktsii”, Problemy kibernetiki, 37, Nauka, M., 1980, 5–17 | MR

[10] Minskii M., Peipert S., Perseptrony, Mir, M., 1971

[11] Rozenblatt F., Printsipy neirodinamiki. Pertseptron i teoriya mekhanizmov mozga, Mir, M., 1965

[12] Cybenko G., “Approximations by superpositions of sigmoidal functions”, Math. Control Signals Systems, 2 (1989), 303–314 | DOI | MR | Zbl

[13] Funahashi K., “On the approximate realization of continuous mappings by neural networks”, Neural Networks, 2:3 (1989), 183–192 | DOI

[14] Jackson D., Über die Genauigkeit der Annäherung stetiger Funktionen durch ganze rationale Funktionen gegebenen Grades und trigonometrische Summen gegebener Ordnung, Preisschrift und Inaugural Dissertation, Göttingen, 1911 | Zbl

[15] Jackson D., The Theory of Approximation, Colloq. Publ., 11, Amer. Math. Soc., 1930 | MR

[16] Hecht-Nielsen R., “Kolmogorov's mapping neural network existence theorem”, IEEE First Annual Int. Conf. on Neural Networks, Vol. 3, San Diego, 1987, 11–13

[17] Hornick K., Stinchcombe M., White H., “Multilayer feedforward networks are universal approximators”, Neural Networks, 2:5 (1989), 359–366 | DOI

[18] Slupecki J., “Kryterium pelnosci wielowartosciowych systemow rachunkow zdan”, C. R. Classe III, 32:1–3 (1939), 102–109

[19] White H., Gallant A. R., Hornik K., Stinchcombe M., Wooldridge J., Artificial Neural Networks: Approximation and Learning Theory, Blackwell, Cambridge, MA, 1992 | MR