Geodesic
Higher Order Orthogonal Polynomials as Activation Functions in Artificial Neural Networks
Serdica Journal of Computing, Tome 17 (2023) no. 1, pp. 1-16.

Voir la notice de l'article provenant de la source Bulgarian Digital Mathematics Library

Activation functions are used in Artificial Neural Networks to provide non-linearity to the system. Several different activation functions in use are very well known by almost any AI practitioner however this is not the case for polynomial activation functions. Increasing attention to these valuable mathematical functions can encourage more research and help to fill the gap. During this work, Chebyshev and Hermite orthogonal polynomials were used as activation functions. Calculations were conducted on 3 different datasets with different hyperparameters. According to the results, calculations done by Chebyshev activation functions take less time, but Chebyshev can be more fragile depending on the solved problem. On the other hand, Hermite shows a more robust and generalized behavior, it is less dependent on the problem type, and it improves by necessary adjustments.
Keywords: Activation function, Chebyshev orthogonal polynomials, Hermite orthogonal polynomials, Artificial Neural Networks 
@article{SJC_2023_17_1_a0,
     author = {Nebioglu, Burak and Iliev, Alexander I.},
     title = {Higher {Order} {Orthogonal} {Polynomials} as {Activation} {Functions} in {Artificial} {Neural} {Networks}},
     journal = {Serdica Journal of Computing},
     pages = {1--16},
     publisher = {mathdoc},
     volume = {17},
     number = {1},
     year = {2023},
     language = {en},
     url = {http://geodesic.mathdoc.fr/item/SJC_2023_17_1_a0/}
}
TY  - JOUR
AU  - Nebioglu, Burak
AU  - Iliev, Alexander I.
TI  - Higher Order Orthogonal Polynomials as Activation Functions in Artificial Neural Networks
JO  - Serdica Journal of Computing
PY  - 2023
SP  - 1
EP  - 16
VL  - 17
IS  - 1
PB  - mathdoc
UR  - http://geodesic.mathdoc.fr/item/SJC_2023_17_1_a0/
LA  - en
ID  - SJC_2023_17_1_a0
ER  - 
%0 Journal Article
%A Nebioglu, Burak
%A Iliev, Alexander I.
%T Higher Order Orthogonal Polynomials as Activation Functions in Artificial Neural Networks
%J Serdica Journal of Computing
%D 2023
%P 1-16
%V 17
%N 1
%I mathdoc
%U http://geodesic.mathdoc.fr/item/SJC_2023_17_1_a0/
%G en
%F SJC_2023_17_1_a0
Nebioglu, Burak; Iliev, Alexander I. Higher Order Orthogonal Polynomials as Activation Functions in Artificial Neural Networks. Serdica Journal of Computing, Tome 17 (2023) no. 1, pp. 1-16. http://geodesic.mathdoc.fr/item/SJC_2023_17_1_a0/