Geodesic
Generalized approximation theorem and computational capabilities of neural networks
Sibirskij žurnal vyčislitelʹnoj matematiki, Tome 1 (1998) no. 1, pp. 11-24.

Voir la notice de l'article provenant de la source Math-Net.Ru

Computational capabilities of artificial neural networks are studied. In this connection comes up the classical problem on representation of function of several variables by means of superpositions and sums of functions of one variable, and appears a new edition of this problem (using only one arbitrarily chosen nonlinear function of one variable). It has been shown that it is possible to obtain arbitrarily exact approximation of any continuous function of several variables using operations of summation and multiplication by number, superposition of functions, linear functions and one arbitrary continuous nonlinear function of one variable. For polynomials an algebraic variant of the theorem is proved. For neural networks the obtained results mean that the only requirement for activation function of neuron is nonlinearity – and nothing else. 
@article{SJVM_1998_1_1_a2,
     author = {A. N. Gorban'},
     title = {Generalized approximation theorem and computational capabilities of neural networks},
     journal = {Sibirskij \v{z}urnal vy\v{c}islitelʹnoj matematiki},
     pages = {11--24},
     publisher = {mathdoc},
     volume = {1},
     number = {1},
     year = {1998},
     language = {ru},
     url = {http://geodesic.mathdoc.fr/item/SJVM_1998_1_1_a2/}
}
TY  - JOUR
AU  - A. N. Gorban'
TI  - Generalized approximation theorem and computational capabilities of neural networks
JO  - Sibirskij žurnal vyčislitelʹnoj matematiki
PY  - 1998
SP  - 11
EP  - 24
VL  - 1
IS  - 1
PB  - mathdoc
UR  - http://geodesic.mathdoc.fr/item/SJVM_1998_1_1_a2/
LA  - ru
ID  - SJVM_1998_1_1_a2
ER  - 
%0 Journal Article
%A A. N. Gorban'
%T Generalized approximation theorem and computational capabilities of neural networks
%J Sibirskij žurnal vyčislitelʹnoj matematiki
%D 1998
%P 11-24
%V 1
%N 1
%I mathdoc
%U http://geodesic.mathdoc.fr/item/SJVM_1998_1_1_a2/
%G ru
%F SJVM_1998_1_1_a2
A. N. Gorban'. Generalized approximation theorem and computational capabilities of neural networks. Sibirskij žurnal vyčislitelʹnoj matematiki, Tome 1 (1998) no. 1, pp. 11-24. http://geodesic.mathdoc.fr/item/SJVM_1998_1_1_a2/

[1] Zurada J. M., Introduction to artificial neural systems, PWS Publ. Co., 1992

[2] Haykin S., Neural networks. A comprehensive foundations, McMillan College Publ. Co., N.Y., 1994

[3] Gorban A. N., Rossiev D. A., Neironnye seti na personalnom kompyutere, Nauka, Novosibirsk, 1996 | MR

[4] Le Cun Y., Denker J. S., Solla S. A., “Optimal brain damage”, Advances in Neural Information Processing Systems II (Denver 1989), Morgan Kaufman, San Mateo, 1990, 598–605

[5] Gorban A. N., Obuchenie neironnykh setei, ParaGraph, M., 1990; Training neural networks, Neurocomputing, 6, Scientific Siberian A. AMSE Trans., 1993, 134 pp.

[6] Prechelt L., “Comparing adaptive and non-adaptive connection pruning with pure early stopping”, Progress in Neural Information Processing (Hong Kong, September 24–27, 1996), v. 1, Springer, 1996, 46–52

[7] Cybenko G., “Approximation by superposition of a sigmoidal function”, Mathematics of Control, Signals, and Systems, 2 (1989), 303–314 | DOI | MR | Zbl

[8] Hornik K., Stinchcombe M., White H., “Multilayer feedforward networks are universal approximators”, Neural Networks, 2 (1989), 359–366 | DOI

[9] Kochenov D. A., Rossiev D. A., “Approximations of functions of C[A,B] class by neural-net predictors (architectures and results)”, Neurocomputing, 6, Scientific Siberian A. AMSE Trans., 1993, 189–203

[10] Gilev S. E., Gorban A. N., “On completness of the class of functions computable by neural networks”, Proc. of the World Congress on Neural Networks (WCNN'96) (Sept. 15–18, 1996, San Diego), Lawrens Erlbaum Accociates, CA, 1996, 984–991

[11] Kolmogorov A. N., “O predstavlenii nepreryvnykh funktsii neskolkikh peremennykh superpozitsiyami nepreryvnykh funktsii menshego chisla peremennykh”, Dokl. AN SSSR, 108:2 (1956), 179–182 | MR

[12] Arnold V. I., “O funktsiyakh trekh peremennykh”, Dokl. AN SSSR, 114:4 (1957), 679–681 | MR

[13] Kolmogorov A. N., “O predstavlenii nepreryvnykh funktsii neskolkikh peremennykh v vide superpozitsii nepreryvnykh funktsii odnogo peremennogo”, Dokl. AN SSSR, 114:5 (1957), 953–956 | MR

[14] Vitushkin A. G., O mnogomernykh variatsiyakh, Fizmatgiz, M., 1955

[15] Stone M. N., “The generalized Weierstrass approximation theorem”, Math. Mag., 21 (1948), 167–183 ; 237–254 | DOI | MR

[16] Shefer X., Topologicheskie vektornye prostranstva, Mir, M., 1971 | MR