Temporal correlation theory and modeling the segmentation of the visual information in the brain (a~review)

Ya. B. Kazanovich

Matematičeskaâ biologiâ i bioinformatika, Tome 5 (2010) no. 1, pp. 43-97.

Résumé

A review of neural network models for image segmentation is presented. The models are based on dynamic processes, oscillations, and synchronization of neural activity. Neurobiological principles of the models, the mechanisms of their functioning, and the results of simulations are described. The performance of the models and the perspectives of improving the segmentation quality are discussed.

Bibliographie

[1] Baddeley A., “Exploring the central executive”, Quarterly Journal of Experimental Psychology A, 49 (1996), 5–28 | DOI

[2] Baddeley A., “Fractionating the central executive”, Principles of Frontal Lobe Function, eds. Stuss D. and Knight R. T., Oxford University Press, New York, 2002, 246–260

[3] Borisyuk R., Kazanovich Y., “Oscillatory model of attention-guided object selection and novelty detection”, Neural Networks, 17 (2004), 899–915 | DOI | Zbl

[4] Borisyuk R., Kazanovich Y., Chik D., Tikhanoff V., Cangelosi A., “A neural model of selective attention and object segmentation in the visual scene: An approach based on partial synchronization and star-like architecture of connections”, Neural Networks, 22 (2009), 707–719 | DOI

[5] Broussard R. P., Rogers S. K., Oxley M. E., Tarr G. L., “Physiologically motivated image fusion for object detection using a pulse coupled neural network”, IEEE Trans. Neural Networks, 10 (1999), 554–563 | DOI

[6] Buhmann J. M., Lange T., Ramacher U., “Image segmentation by networks of spiking neurons”, Neural Computation, 17 (2005), 1010–1031 | DOI | Zbl

[7] Caelli T., Bischof W., Machine learning and image interpretation, Plenum Press, New York, 1997

[8] Carson C., Belongie S., Greenspan H., Malik J., “Blobworld: Image segmentation using expectation-maximization and its application to image querying”, IEEE Trans. Pattern Analysis and Machine Intelligence, 24 (2002), 1026–1038 | DOI

[9] Chen K., Wang D. L., “A dynamically coupled neural oscillator networks for image segmentation”, Neural Network, 15 (2002), 423–439 | DOI

[10] Chen K., Wang D. L., Liu X., “Weight adaptation and oscillatory correlation for image segmentation”, IEEE Trans. Neural Networks, 11 (2000), 1106–1123 | DOI

[11] Chik D., Borisyuk R., Kazanovich Y., “Selective attention model with spiking elements”, Neural Networks, 22 (2009), 890–900 | DOI

[12] Choe Y., Miikkulainen R., “Self-organization and segmentation in a laterally connected orientation map of spiking neurons”, Neurocomputing, 21 (1998), 139–157 | DOI | Zbl

[13] Choe Y., Miikkulainen R., “Contour integration and segmentation with self-organized lateral connections”, Biol. Cybern., 90 (2004), 75–88 | DOI | Zbl

[14] Cowan N., “Evolving conceptions of memory storage, selective attention and their mutual constraints within the human information processing system”, Psychological Bulletin, 104 (1988), 163–191 | DOI

[15] Doesburg S. M., Roggeveen A. B., Kitajo K., Ward L. M., “Large-scale gamma-band phase synchronization and selective attention”, Cerebral Cortex, 18 (2008), 386–396 | DOI

[16] Eckhorn R., Bauer R., Jordon W., Brosch M., Kruse W., Munk M., Reitboeck H. J., “Coherent oscillations: a mechanism of feature linking in the visual cortex”, Biol. Cybern., 60 (1988), 121–130 | DOI

[17] Eckhorn R., Reitboeck H. J., Amdt M., Dicke P., “Feature linking via synchronization among distributed assemblies: Simulation of results for cat visual cortex”, Neural Computation, 2 (1990), 293–307 | DOI

[18] Engel A., König P., Singer W., “Direct physiological evidence for scene segmentation by temporal coding”, Proc. Natl. Acad. Sci. (USA), 88 (1991), 9136–9140 | DOI

[19] Fries P., Reynolds J., Rorie A., Desimone R., “Modulation of oscillatory neuronal synchronization by selective visual attention”, Science, 291 (2001), 1560–1563 | DOI

[20] Fries P., Schroeder J.-H., Roelfsema P. R., Singer W., Engel A. K., “Oscillatory neural synchronization in primary visual cortex as a correlate of stimulus selection”, J. Neurosci., 22 (2002), 3739–3754

[21] Gray C. M., “Synchronous oscillations in neural systems: Mechanisms and functions”, J. Comput. Neurosci., 1 (1994), 11–38 | DOI

[22] Gray C. M., “The temporal correlation hypothesis of visual feature integration is still alive and well”, Neuron, 24 (1999), 31–47 | DOI

[23] Gray C. M., Konig P., Engel A. K., Singer W., “Oscillatory responses in cat visual cortex exhibit intercolumnar synchronization which reflects global stimulus properties”, Nature, 388 (1989), 334–337 | DOI

[24] Gregoriou G. G., Gotts S. J., Zhou H., Desimone R., “High-frequency, long-range coupling between prefrontal and visual cortex during attention”, Science, 324 (2009), 1207–1210 | DOI

[25] Haralick R. M., Shapiro L. G., “Image segmentation techniques”, Computer Vision, Graphics and Image Processing, 29 (1985), 100–132 | DOI

[26] Herrman C. S., Munk M. H. J., Engel A. K., “Cognitive functions of gamma-band activity: memory match and utilization”, Trends in Cognitive Sciences, 8 (2004), 347–355 | DOI

[27] Itti L., Koch C., “A saliency-based search mechanism for overt and covert shifts of visual attention”, Vision Research, 40 (2000), 1489–1506 | DOI

[28] Itti L., Koch C., “Computational modeling of visual attention”, Nature Reviews Neuroscience, 2 (2001), 194–203 | DOI

[29] Jin Y., Angelini E., Laine A., “Wavelets in Medical Image Processing: Denoising, Segmentation, and Registration”, Handbook of Biomedical Image Analysis, part A, v. 1, Topics in Biomedical Engineering. International Book Series, Segmentation models, eds. Suri J. S., Wilson D. L., Laxminarayan S., Springer, 2005, 305–358

[30] Jutras M. J., “Gamma-band synchronization in the macaque hippocampus and memory formation”, J. Neurosci., 29 (2009), 12521–12531 | DOI

[31] Kaneko K., “Clustering, coding switching, hierarchical ordering, and coding in a network of chaotic elements”, Physica D, 41 (1990), 137–172 | DOI | MR | Zbl

[32] Kapadia M. K., Ito M., Gilbert C. D., Westheimer G., “Improvement in visual sensitivity by changes in local context: parallel studies in human observers and in V1 of alert monkeys”, Neuron, 15 (1995), 843–856 | DOI

[33] Kazanovich Y., Borisyuk R., “Object selection by an oscillatory neural network”, BioSystems, 67 (2002), 103–111 | DOI

[34] Klein R., “Inhibitory tagging system facilitates visual search”, Nature, 334 (1988), 430–431 | DOI

[35] Knoblauch A., Palm G., “Pattern separation and synchronization in spiking associative memories and visual areas”, Neural Networks, 14 (2001), 763–780 | DOI

[36] Knoblauch A., Palm G., “Scene segmentation by spike synchronization in reciprocally connected visual areas. I. Local effects of cortical feedback”, Biol. Cybern., 87 (2002), 151–167 | DOI | Zbl

[37] Knoblauch A., Palm G., “Scene segmentation by spike synchronization in reciprocally connected visual areas. II. Global assemblies and synchronization on larger space and time scales”, Biol. Cybern., 87 (2002), 168–184 | DOI | Zbl

[38] Kryukov V. I., “An attention model based on the principle of dominanta”, Neurocomputers and attention, v. I, Neurobiology, synchronisation and chaos, eds. Holden A. V., Kryukov V. I., Univ. Press, Manchester, 1991, 319–351 | MR

[39] Kuzmina M., Manykin E., Surina I., “Oscillatory networks with self-organized dynamical connections for synchronization-based image segmentation”, BioSystems, 76 (2004), 43–53 | DOI

[40] Labbi A., Milanese R., Bosch H., “Visual object segmentation using FitzHugh–Nagumo oscillators”, Nonlinear Analysis. Theory, Methods Applications, 47 (2001), 5827–5838 | DOI | MR | Zbl

[41] Li Z., “A neural model of contour integration in the primary visual cortex”, Neural Computation, 10 (1998), 903–940 | DOI

[42] Li Z., “Visual segmentation by contextual influences via intra-cortical interactions in the primary visual cortex”, Network, 10 (1999), 187–212 | DOI

[43] Li Z., “Pre-attentive segmentation in the primary visual cortex”, Spatial Vision, 13 (2000), 25–50 | DOI

[44] Li Z., “Computational design and nonlinear dynamics of a recurrent network model of the primary visual cortex”, Neural Computation, 13 (2001), 1749–1780 | DOI | Zbl

[45] Li J., Gray R. M., Image Segmentation and Compression Using Hidden Markov Models, The International Series in Engineering and Computer Science, Klawer Academic Publishers, 2000

[46] Lindblad T., Kinser J. M., Image processing using pulse-coupled neural networks, 2nd ed., Springer-Verlag, 2005

[47] Malsburg C. von der., The correlation theory of brain function, Internal report 81-2, Max-Plank Institute for Biophysical Chemistry, 1981; Models of Neural Networks, eds. Domany E., van Hemmen J. L., Schulten K., Springer, New York, 1994, 95–119 | MR

[48] Miltner W., Braun C., Arnold M., Witte H., Taub E., “Coherence of gamma-band EEG activity as a basis for associative learning”, Nature, 397 (1999), 434–436 | DOI

[49] Mirollo R. E., Strogatz S. H., “Synchronization of pulse-coupled biological oscillators”, SIAM J. Applied Mathematics, 50 (1990), 1645–1662 | DOI | MR | Zbl

[50] Moran J., Desimone R., “Selective attention gates visual processing in the extrastriate cortex”, Science, 229 (1985), 782–784 | DOI

[51] Itti G., Rees G., Tsotsos J. K. (eds.), Neurobiology of Attention, Elsevier, Amsterdam, 2005

[52] Opara R., Wörgötter F., “Using visual latencies to improve image segmentation”, Neural Computation, 8 (1996), 1493–1520 | DOI

[53] Opara R., Wörgötter F., “A fast and robust cluster update algorithm for image segmentation in spine-lattice models without annealing – Visual latencies revisited”, Neural Computation, 10 (1998), 1547–1566 | DOI

[54] Pal N. R., Pal S. K., “A review on image segmentation techniques”, Pattern Recognition, 26 (1993), 1277–1294 | DOI

[55] Palm G., Knoblauch A., “Scene segmentation through synchronization”, Neurobiology of Attention, eds. Itti L., Rees G., and Tsotsos J. K., Elsevier, San Diego. CA, 2005, 618–623

[56] Pham D. L., Xu C., Prince J. L., “Current methods in medical image segmentation”, Annual Review of Biomedical Engineering, 2 (2000), 315–337 | DOI | MR

[57] Polakowski W., Rogers S., Ruck D., Hoffmeister J., Raines R., “Computer aided breast cancer detection and diagnosis of masses using difference of gaussians and derivativebased feature saliency”, IEEE Trans. Med. Imag., 16 (1997), 811–819 | DOI

[58] Reynolds J., Desimone R., “The role of neural mechanisms of attention in solving the binding problem”, Neuron, 24 (1999), 19–29 | DOI | MR

[59] Rhouma M. B. H., Frigui H., “Self-organization of pulsed-coupled oscillators with application to clustering”, IEEE Trans. Pattern Analysis and Machine Intelligence, 23 (2001), 180–195 | DOI

[60] Rogers S. K., Colombi J. M., Martin C. E., Gainey J. C., Fielding K. H., Burns T. J., Ruck D. W., Kabrisky M., Oxley M., “Neural networks for automatic target recognition”, IEEE Trans. Neural Networks, 8 (1995), 1153–1184

[61] Rutishauser U., Ross I. B., Mamelak A. N., Schuman E. M., “Human memory strength is predicted by theta-frequency phase-locking of single neurons”, Nature, 464 (2010), 903–907 | DOI

[62] Sauseng P., Klimesch W., Gruber W. R., Birbaumer N., “Cross-frequency phase synchronization: a brain mechanism of memory matching and attention”, Neuroimage, 40 (2008), 308–317 | DOI

[63] Schillen T. B., Konig P., “Binding by temporal structure in multiple feature domains of an oscillatory neural network”, Biol. Cybern., 70 (1994), 397–405 | DOI

[64] Sederberg P. B., Kahana M. J., Howard M. W., Donner E. J., Madsen J. R., “Theta and gamma oscillations during encoding predict subsequent recall”, J. Neurosci., 23 (2003), 10809–10814

[65] Shapiro L. G., Stockman G. C., Computer Vision, Prentice-Hall, New Jersey, 2001

[66] Sharon B., Galun M., Sharon D., Bash R., Brandt A., “Hierarchy and adaptivity in segmenting visual scenes”, Nature, 442 (2006), 810–813 | DOI

[67] Singer W., “Neuronal synchrony: A versatile code for the definition of relations”, Neuron, 24 (1999), 49–65 | DOI

[68] Singer W., Gray C. M., “Visual feature integration and the temporal correlation hypothesis”, Ann. Rev. Neurosci., 18 (1995), 555–586 | DOI

[69] Steinmetz P. N., Roy A., Fitzgerald P., Hsiao S. S., Johnson K. O., Niebur E., “Attention modulates synchronized neuronal firing in primate somatosensory cortex”, Nature, 404 (2000), 187–190 | DOI

[70] Strogatz S., Nonlinear Dynamics and Chaos, Perseus Publishing, 2000

[71] Ursino M., La Cara G. E., “Modeling segmentation of a visual scene via neural oscillators: fragmentation, discovery of details and attention”, Network, 15 (2004), 69–89 | DOI

[72] Ursino M., La Cara G. E., Sarti A., “Binding and segmentation of multiple objects through neural oscillators inhibited by contour information”, Biol. Cybern., 89 (2003), 56–70 | Zbl

[73] Verschure P., Konig P., “On the role of biophysical properties of cortical neurons in binding and segmentation of visual scenes”, Neural Computation, 11 (1999), 1113–1138 | DOI

[74] Wang D. L., “The time dimension for scene analysis”, IEEE Trans. Neural Networks, 16 (2005), 1401–1426 | DOI

[75] Wang D. L., Terman D., “Locally excitatory globally inhibitory oscillator network”, IEEE Trans. Neural Networks, 6 (1995), 283–286 | DOI

[76] Wang D. L., Terman D., “Image segmentation based on oscillatory correlation”, Neural Computation, 9 (1997), 805–836 | DOI

[77] Weickert J., “A review of nonlinear diffusion filtering”, Scale-space theory in computer vision, eds. Romany B., Florack L., Koenderick J., Viergever M., Springer, Berlin, 1997, 3–28

[78] Wikipedia-Segmentation (image processing)., , (data obrascheniya: 28.06.2010) http://en.wikipedia.org/wiki/ Segmentation_(image_processing)

[79] Willshaw D., Buneman O., Longuet-Higgins H., “Non-holographic associative memory”, Nature, 222 (1969), 960–962 | DOI

[80] Womelsdorf T., Schoffelen J. M., Oostenveld R., Singer W., Desimone R., Engel A. K., Fries P., “Modulation of neuronal interactions through neuronal synchronization”, Science, 316 (2007), 1609–1612 | DOI

[81] Yu B., Zhang L., “Pulse-coupled neural networks for contour and motion matching”, IEEE Trans. Neural Networks, 15 (2004), 1186–1201 | DOI

[82] Zhang A., “A survey on evaluation methods for image segmentation”, Pattern Recognition, 29 (1996), 1235–1346

[83] Zhao L., de Carvalho A., Li Z., “Pixel clustering by adaptive pixel moving and chaotic synchronization”, IEEE Trans. on Neural Networks, 15 (2004), 1176–1185 | DOI

[84] Zhao L., Furukawa R. A., de Carvalho A., “A network of coupled chaotic maps for adaptive multy-scale image segmentation”, Int. J. Neural Systems, 13 (2003), 129–137 | DOI

[85] Zhao L., Macau E., “A network of dynamically coupled chaotic maps for scene segmentation”, IEEE Trans. on Neural Networks, 12 (2001), 1375–1385 | DOI

[86] Zhu S. C., Yuille Y., “Region competition: Unifying snakes, region growing, and Bayes/MDL for multiband image segmentation”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 18 (1996), 884–900 | DOI

[87] Abarbanel G. D. I., Rabinovich M. I., Selverston A., i dr., “Sinkhronizatsiya v neironnykh ansamblyakh”, Uspekhi fizicheskikh nauk, 166:4 (1996), 363–390

[88] Borisyuk G. N., Borisyuk R. M., Kazanovich Ya. B., Ivanitskii G. R., “Modelirovanie dinamiki neironnoi aktivnosti i obrabotka informatsii v mozge – itogi “desyatiletiya””, Uspekhi fizicheskikh nauk, 172:10 (2002), 1189–1214

[89] Vezhnevets A., Barinova O., Metody segmentatsii izobrazhenii: avtomaticheskaya segmentatsiya. Grafika i multimedia, (data obrascheniya: 28.06.2010) http://cgm.computergraphics.ru/content/view/147

[90] Grichuk S., Kuzmina M. G., Manykin E. A., “Selektivnaya segmentatsiya realnykh izobrazhenii ostsillyatorno-setevym metodom”, Neiroinformatika-2007, IX Vserossiiskaya nauchno-tekhnicheskaya konferentsiya, v. 3, eds. O. A. Mishulina, MIFI, Moskva, 2007, 235–244

[91] Grichuk S., Kuzmina M. G., Manykin E. A., “Vydelenie ob'ektov v zritelnoi stsene ostsillyatorno-setevym metodom”, Neiroinformatika-2009, XI Vserossiiskaya nauchno-tekhnicheskaya konferentsiya, v. 3, eds. O. A. Mishulina, MIFI, Moskva, 2009, 167–172

[92] Kazanovich Ya. B., Shmatchenko V. V., “Ostsillyatornye neirosetevye modeli segmentatsii izobrazhenii i zritelnogo vnimaniya”, Neiroinformatika-2004. Lektsii po neiroinformatike, VI Vserossiiskaya nauchno-tekhnicheskaya konferentsiya, v. 1, eds. Tyumentsev Yu. V., MIFI, Moskva, 2004, 72–125

[93] Kryukov V. I., “Model vnimaniya i pamyati, osnovannaya na printsipe dominanty i komparatornoi funktsii gippokampa”, Zhurnal vysshei nervnoi deyatelnosti, 54:1 (2004), 10–29

[94] Kuzmina M. G., Manykin E. A., “Ostsillyatornaya set dlya segmentatsii izobrazhenii: novye razrabotki”, Neiroinformatika-2004, VII Vserossiiskaya nauchno-tekhnicheskaya konferentsiya, v. 1, eds. Mishulina O. A., MIFI, Moskva, 2004, 262–268

[95] Chik D., Borisyuk R. M., Kazanovich Ya. B., “Rezhimy sinkhronizatsiya v seti neironov Khodzhkina–Khaksli s tsentralnym elementom”, Matematicheskaya biologiya i bioinformatika (elektronnyi zhurnal), 3:1 (2008), 16–35, (data obrascheniya: 28.06.2010) http://www.matbio.org/downloads/Chick2008(3_16).pdf