Geodesic
Convergence of the partition function in the static word embedding model
Eurasian mathematical journal, Tome 13 (2022) no. 4, pp. 70-81

Voir la notice de l'article provenant de la source Math-Net.Ru

We develop an asymptotic theory for the partition function of the word embeddings model WORD2VEC. The proof involves a study of properties of matrices, their determinants and distributions of random normal vectors when their dimension tends to infinity. The conditions imposed are mild enough to cover practically important situations. The implication is that for any word $i$ from a vocabulary $\mathcal{W}$, the context vector $\mathbf{c}_i$ is a reflection of the word vector $\mathbf{w}_i$ in approximately half of the dimensions. This allows us to halve the number of trainable parameters in static word embedding models. 
@article{EMJ_2022_13_4_a5,
     author = {K. Mynbaev and Zh. Assylbekov},
     title = {Convergence of the partition function in the static word embedding model},
     journal = {Eurasian mathematical journal},
     pages = {70--81},
     publisher = {mathdoc},
     volume = {13},
     number = {4},
     year = {2022},
     language = {en},
     url = {http://geodesic.mathdoc.fr/item/EMJ_2022_13_4_a5/}
}
TY  - JOUR
AU  - K. Mynbaev
AU  - Zh. Assylbekov
TI  - Convergence of the partition function in the static word embedding model
JO  - Eurasian mathematical journal
PY  - 2022
SP  - 70
EP  - 81
VL  - 13
IS  - 4
PB  - mathdoc
UR  - http://geodesic.mathdoc.fr/item/EMJ_2022_13_4_a5/
LA  - en
ID  - EMJ_2022_13_4_a5
ER  - 
%0 Journal Article
%A K. Mynbaev
%A Zh. Assylbekov
%T Convergence of the partition function in the static word embedding model
%J Eurasian mathematical journal
%D 2022
%P 70-81
%V 13
%N 4
%I mathdoc
%U http://geodesic.mathdoc.fr/item/EMJ_2022_13_4_a5/
%G en
%F EMJ_2022_13_4_a5
K. Mynbaev; Zh. Assylbekov. Convergence of the partition function in the static word embedding model. Eurasian mathematical journal, Tome 13 (2022) no. 4, pp. 70-81. http://geodesic.mathdoc.fr/item/EMJ_2022_13_4_a5/