Counting hidden neural networks

Richard, Anthony ; Desrosiers, Patrick ; Hardy, Simon ; Doyon, Nicolas

Journal of integer sequences, Tome 19 (2016) no. 4.

Résumé

Summary: We apply combinatorial tools, including Pólya's theorem, to enumerate all possible networks for which (1) the network contains distinguishable input and output nodes as well as partially distinguishable intermediate nodes; (2) all connections are directed and for each pair of nodes, there are at most two connections, that is, at most one connection per direction; (3) input nodes send connections but don't receive any, while output nodes receive connections but don't send any; (4) every intermediate node receives a path from an input node and sends a path to at least one output node; and (5) input nodes don't send direct connections to output nodes. We first obtain the generating function for the number of such networks, and then use it to obtain precise estimates for the number of networks. Finally, we develop a computer algorithm that allows us to generate these networks. This work could become useful in the field of neuroscience, in which the problem of deciphering the structure of hidden networks is of the utmost importance, since there are several instances in which the activity of input and output neurons can be directly measured, while no direct access to the intermediate network is possible. Our results can also be used to count the number of finite automata in which each cell plays a relevant role.

Détail

Keywords: graphical enumeration, Pólya's enumeration theorem, digraph enumeration, hidden neural networks, finite automata enumeration