Machine Learning for Natural Language Processing: Contemporary State

X. A. Naidenova ; O. A. Nevzorova

Učënye zapiski Kazanskogo universiteta. Seriâ Fiziko-matematičeskie nauki, Kazanskii Gosudarstvennyi Universitet. Uchenye Zapiski. Seriya Fiziko-Matematichaskie Nauki, Tome 150 (2008) no. 4, pp. 5-24

Résumé

Contemporary methods of machine learning used for natural language processing tasks are discussed in the paper. Various classic tasks of natural language processing are considered, including the tasks of morphological and syntactical analysis. Special attention is given to the methods of machine learning used for construction of ontological models.

Détail

Keywords: methods of machine learning, natural language processing, morphology, semantics, ontology.
Mots-clés : syntax

Bibliographie

[1] Abney S., “Partial Parsing via Finite-State Cascades”, ESSLLI' 96 Workshop on Robust Parsing Workshop, Prague, Czech Republic, 1996, 71–84

[2] Abney S., “Chunks and Dependencies: Bringing Processing Evidence to Bear on Syntax”, Computational Linguistics and the Foundations of Linguistic Theory, eds. J. Cole, G. M. Green, J. L. Morgan, CSLI Publications, Stanford, CA, 1995, 145–164

[3] Abney S., “Parsing by Chunks”, Principle-Based Parsing, eds. R. Berwick, S. Abney, C. Tenny, Kluwer Acad. Publ., Dordrecht, The Netherlands, 1991, 257–278

[4] D. W. Aha (ed.), Lazy Learning, Kluwer Acad. Publ., Dordrecht, The Netherlands, 1997, 625 pp.

[5] Allen J., Natural Language Understanding, Benjamin/Cummings Publishing Company, Menlo Park, CA, 1995, 625 pp.

[6] Andr'e E., Binsted K., Tanaka-Ishii K., Luke S., Herzog G., Rist T., “Three RoboCup simulation league commentator systems”, AI Magazine, 21:1 (2000), 57–66

[7] Basili R., Pazienza M. T., Velardi P., “Hierarchical clustering of verbs”, Proc. of the Workshop on Acquisition of Lexical Knowledge from Text, 1993, 70–81

[8] Bisson G. , Nedellec C., Canamero L., “Designing clustering methods for ontology building: The Mo'K workbench”, Proc. of the ECAI Ontology Learning Workshop, 2000, 13–18

[9] Brill E., “Some advances in transformation-based part-of-speech tagging”, Proc. of the Nat. Conf. on AI (AAAI), 1994, 722–727

[10] Buitelaar P., Olejnik D., Sintek M., “A Protégé plug-in for ontology extraction from text based on linguistic analysis”, Proc. of the 1st European Semantic Web Symposium (ESWS), 2004, 31–44

[11] Bunescu C., Mooney J., “Extracting Relations from Text: From Word Sequences to Dependency Paths”, Natural Language Processing and Text Mining, eds. A. Rao, S. R. Poteet, Springer, 2007, 29–44

[12] Carreras X., M'arquez L., “Introduction to the CoNLL–2005 shared task: Semantic role labeling”, Proc. of the 9th Conf. on Natural Language Learning (CoNLL–2005), 2005, 152–164

[13] Cestnik B., Kononenko I., Bratko I., “ASSISTANT–86: A Knowledge-Elicitation Tool for Sophisticated Users”, Progress in Machine Learning, eds. I. Bratko, N. Lavrac, Sigma Press, Wilmslow, 1987

[14] Charniak E., Hendrickson C., Jacobson N., Perkowitz M., “Equations for part-of-speech tagging”, Proc. of the 11th Nat. Conf. on AI (AAAI), 1993, 784–789

[15] Charniak E., Berland M., “Finding parts in very large corpora”, Proc. of the 37th Annual Meeting of the ACL, 1999, 57–64

[16] Chen M., Foroughi E., Heintz F., Kapetanakis S., Kostiadis K., Kummeneje J., Noda I., Obst O., Riley P., Steffens T., Wang Y., Yin X., Users manual: RoboCup soccer server manual for soccer server version 7.07 and later, , 2003 http://sourceforge.net/projects/sserver/

[17] Ciaramita M., Gangemi A., Ratsch E., Šaric J., Rojas I., “Unsupervised learning of semantic relation between concepts of a molecular biology ontology”, Proc. of the 19th Int. Joint Conf. on AI (IJCAI), 2005, 659–664

[18] Cimiano P., Ontology Learning and Population from Text. Algorithms, Evaluation and Applications, Springer, 2006, 347 pp. | Zbl

[19] Cimiano P., Holto A., Staab S., “Learning Concept Hierarchies from Text Corpora Using Formal Concept Analysis”, J. Artif. Intellig. Res., 24 (2005), 305–339 | Zbl

[20] Cimiano R., Handschuh S., Staab S., “Towards the self-annotating web”, Proc. of the 13th World Wide Web Conf., 2004, 462–471

[21] Cimiano P., Staab S., Tane J., “Automatic acquisition of taxonomies from text: FCA meet NLP”, Proc. of the PKDD/ECML' 03 Int. Workshop on Adaptive Text Extraction and Mining (ATRM), 2003, 10–17

[22] Cimiano P., Handschuh S., “Ontology-based linguistics annotation”, Proc. of the ACL 2003 workshop on Linguistic annotation: getting the model right, V. 19, 2003, 14–21

[23] Clark S., Weir D., “Class-based probability estimation using a semantic hierarchy”, Comput. Linguist., 28:2 (2002), 187–206 | DOI

[24] Collins M. J., “Three generative, lexicalized models for statistical parsing”, Proc. of the 35th Annual Meeting of the Association for Computational Linguistics (ACL–97), 1997, 16–23

[25] Cristianini N., Shawe-Taylor J., An Introduction to Support Vector Machines and Other Kernel-based Learning Methods, Cambridge Univ. Press, Cambridge, 2000, 204 pp.

[26] van Delden S., Gomez F., “Combining Finite State Automata and Greedy Learning Algorithm tj Determine the Syntactic Roles of Commas”, Proc. of 14th IEEE Int. Conf. on Tools and AI (ICTAI' 02), 2002, 293

[27] Faure D., Nedellec C., “A corpus-based conceptual clustering method for verb frames and ontology”, Proc. of the 1st LREC Workshop on Adapting lexical and corpus resources to sublanguages and applications, Grenada, Spain, 1998, 1–8

[28] Fern A., Givan R., Siskind J. M., “Specific-to-general learning for temporal events with application to learning event definitions from video”, J. Artif. Intellig. Res., 17 (2002), 379–449 | MR | Zbl

[29] Fleischman M., Roy D., “Intentional context in situated natural language learning”, Proc. of 9th Conf. on Computational Natural Language Learning (CoNLL–2005), 2005, 104–111

[30] Gamallo P., Gonzalez M., Agustini A., Lopes G., de Lima V. S., “Mapping syntactic dependencies onto semantic relations”, Proc. of the ECAI Workshop on Machine Learning and NLP for Ontology Engineering, 2002, 15–22

[31] Ganter B., Wille R., Formal Concept Analysis: Mathematical Foundations, Springer-Verlag, New York, 1999, 294 pp. | MR | Zbl

[32] Ge R., Mooney R. J., “Discriminative reranking for semantic parsing”, Proc. of Joint Conf. of the Int. Committee on Computational Linguistics and the Association for Computational Linguistics (COLING–ACL–06), Sydney, Australia, 2006, 263–270

[33] Ge R., Mooney R. J., “A statistical semantic parser that integrates syntax and semantics”, Proc. of the Ninth Conf. on Computational Natural Language Learning (CoNLL–2005), 2005, 9–16

[34] Gildea D., Palmer M., “Automatic labeling of semantic roles”, Comput. Linguist., 23 (2002), 245–248 | DOI

[35] Hahn U., Schnattinger K., “Towards text knowledge engineering”, Proc. of the 15th Nat. Conf. on Artificial Intelligence and the 10th Conf. on Innovative Applications of Artificial Intelligence (AAAI' 98/IAAI' 98), 1998, 524–531

[36] Handschuh S., Staab S., “Authoring and annotation of web pages in CREAM”, Proc. of the 11th Int. World Wide Web Conf. (WWW 2002), Honolulu, Hawaii, 2002, 462–473

[37] Hearst M. A., “Automatic acquisition of hyponyms from large text corpora”, Proc. of 14th Int. Conf. on Computational Linguistics, 1992, 539–545

[38] Haruno M., Shirai S., Ooyama Y., “Using Decision Trees to Construct a Practical Parser”, Machine Learning, 34 (1999), 131–149 | DOI | Zbl

[39] Hindle D., “Noun classification from predicate-argument structures”, Proc. of the Annual Meeting of the Association for Computational Linguistics, 1990, 268–275

[40] Kate R., Mooney R. J., “Using string-kernels for learning semantic parsers”, Proc. of Joint Conf. of the Int. Committee on Computational Linguistics and the Association for Computational Linguistics (COLING–ACL–06), Sydney, Australia, 2006

[41] Kavalec M., Svátek V., “A study on automated relation labeling in ontology learning”, Ontology Learning from Text: Methods, Evaluation and Applications, eds. P. Buitelaar, P. Cimiano, B. Magnini, 2005, 44–58

[42] Kim S.-D., Zhang B.-T., Kim Y. T., “Learning-based Intrasentence Segmentation for Efficient Translation of Long Sentences”, Machine Learning, 16 (2001), 151–174 | Zbl

[43] Kiyavitskaya N., Zeni N., Mich L., Cordy J. R., Mylopoulos J., “Text Mining through Semi Automatic Semantic Annotation”, Proc of PAKM' 2006, 2006; htpp://citeseer.ist.psu.edu/Kiyavitskaya06text.html

[44] Kiyavitskaya N., Zeni N., Mich L., Cordy J. R., Mylopoulos J., “Applying Software Analysis Technology to Lightweight Semantic Markup of Document Text”, Proc. of Int. Conf. on Advances in Pattern Recognition (ICAPR–2005), Bath, UK, 2005, 590–600

[45] Koomen P., Punyakanok V., Roth D., Yih W., “Generalized inference with multiple semantic role labeling systems”, Proc. of the 19th Conf. on Computational Natural Language Learning (CoNLL–2005), 2005, 181–184

[46] Landauer T. K., Foltz P. W., Laham D., “Introduction to Latent Semantic Analysis”, Discourse Processes, 25 (1998), 259–284

[47] Lodhi H., Saunders C., Shawe-Taylor J., Cristianini N., Watkins C., “Text classification using string kernels”, J. Machine Learning Res., 2 (2002), 419–444 | DOI | Zbl

[48] Maedche A., Staab S., Discovering Conceptual Relations from Text, Technical Report 399, Institute AIFB, Karlsruhe University, 2000; \href{http://citeseer.ist.psu.edu/maedche00discovering.html} \allowbreak{http://citeseer.ist.psu.edu/maedche00discovering.html}

[49] Maedche A., Pekar V., Staab S., “Ontology learning. Part one – on discovering taxonomic relations from the web”, Web Intelligence, Springer, 2002

[50] Manning C., Schutze H., Foundations of Statistical Natural Language Processing, The MIT Press, Cambridge, MA, 1999, 620 pp. | MR

[51] Marcus M., Santorini B., Marcinkiewicz M. A., “Building a large annotated corpus of English: The Penn Treebank”, Comput. Linguist., 19:2 (1993), 313–330

[52] McNamara D. C., Levinstein I. B., Boonthum C., “ISTART: Interactive Strategy Training for Active Reading and Thinking”, Behavior Research Methods, Instrument, and Computers, 36 (2004), 222–233

[53] Mooney R. J., “Learning Language from Perceptual Context: A Challenge Problem for AI”, Proc. of the 2006 AAAI Fellows Symposium, Boston, MA, 2006; http://www.cs.utexas.edu/~ml/publication/nl.html

[54] Mustafaraj E., Hoof M., Freisleben B., “Mining Diagnostic Text Reports by Learning to Annotate Knowledge Roles”, Natural Language Processing and Text Mining, eds. A. Rao, S. R. Poteet, Springer, 2007, 45–67

[55] Naidenova X. A., “Reducing a Class of Machine Learning Algorithms to Logical Commonsense Reasoning Operations”, Mathematical Methods for Knowledge Discovery and Data Mining, eds. G. Felici, C. Vercellis, Information Science Reference, Hershey, New York, 2007, Chapter III, 41–64

[56] Naidenova X., “An Incremental Learning Algorithm for Inferring Logical Rules from Examples in the Framework of Common Reasoning Process”, Data Mining and Knowledge Discovery Approaches Based on Rule Induction Techniques, eds. E. Triantaphyllou, G. Felici, Springer, 2006, 89–146

[57] Naidenova X. A., Shagalov V. L., Plaksin M. V., “Inductive Inferring All Good Classification Tests”, Proc. of the Int. Conf. “Knowledge–Dialog–Solution” (KDS–95), V. 1, 1995, 79–84

[58] Popescu A.-M., Etzioni O., “Extracting Product Features and Opinions from reviews”, Natural Language Processing and Text Mining, eds. A. Rao, S. R. Poteet, Springer, 2007, 9–28

[59] Priss U., Old L. J., “Modelling Lexical Databases with Formal Concept Analysis”, J. Univer. Comput. Sci., 10:8 (2004), 967–984

[60] The Penn Treebank, \href{http://www.cis.upenn.edu/~treebank/} \allowbreak{http://www.cis.upenn.edu/~treebank/}

[61] Quinlan J. R., “Induction of Decision Trees”, Machine Learning, 1 (1986), 81–106

[62] Quinlan J. R., C4.5: Programs for Machine Learning, Morgan Kaufmann Publ., San Mateo, CA, 1993, 302 pp.

[63] Rabiner L. R., “A tutorial on hidden Markov models and selected application in speech recognition”, Proc. IEEE, 77:2 (1989), 257–286 | DOI

[64] Resnik P., “Selectional preference and sense disambiguation”, Proc. of the ACL SIGLEX WORKSHOP on Tagging Test with Lexical Semantics: Why? What? And How?, 1997, 52–57

[65] Ribas F., “On learning more appropriate selectional restrictions”, Proc. of the 7th Conf. of the European chapter of the Association for Computational Linguistics (EACL), 1995, 112–118

[66] Schmid H., “Probabilistic part-of-speech tagging using decision trees”, Proc. of Int. Conf. on New Methods in Language Processing, Manchester, UK, 1994, 44–49

[67] Shulte im W., “Clustering verbs semantically according to their alternation behavior”, Proc. of the 18th Int. Conf. on Computational Linguistics (COLING–00), 2000, 747–753

[68] Siskind J. M., “A computational study of cross-situational techniques for learning word-to-meaning mappings”, Cognition, 61:1 (1996), 39–91 | DOI

[69] Shutz A., Buitelaar P., “RelExt: A tool for relation extraction from text in ontology extension”, Proc. of the Int. Semantic Web Conf., 2005, 593–606

[70] Tang L. R., Mooney R. J., “Using multiple clause constructors in inductive logic programming for semantic parsing”, Proc. of the 12th Europ. Conf. on Machine Learnin., Freiburg, Germany, 2001, 466–477 | Zbl

[71] Thompson C. A., Mooney R. J., “Acquiring word-meaning mappings for natural language interfaces”, J. Artif. Intellig. Res., 18 (2003), 1–44 | Zbl

[72] Toutanova K., Haghighi A., Manning C. D., “Joint learning improves semantic role labeling”, Proc. of the 43nd Annual Meeting of the Association for Computational Linguistics (ACL–05), 2005, 589–596

[73] Tufis D., Mason O., “Tagging Romanian Texts: a Case Study for QTAG, a Language Independent Probabilistic Tagger”, Proc. of the 1st Int. Conf. on Language Resources and Evaluation (LREC), 1998, 589–596

[74] Velardi P., Fabriani P., Missikoff M., “Using text processing techniques to automatically enrich a domain ontology”, Proc. of the ACM Int. Conf. on Formal Ontology in Information Systems, 2001, 270–274

[75] Wong Y. W., Learning for semantic parsing using statistical machine translation techniques, Doctoral Dissertation Proposal, University of Texas at Austin; Also appears as Technical Report UT-AI-05-323, Artificial Intelligence Lab, University of Texas at Austin, October, 2005, 53 pp.

[76] Wong Y. W., Mooney R. J., “Learning for semantic parsing with statistical machine translation”, Proc. of the Human Language Technology Conference – North American Chapter of the Association for Computational Linguistics Annual Meeting (HLT/NAACL–06), New York, 2006, 439–446

[77] Wordnet, \href{http://wordnet.princeton.edu/} \allowbreak{http://wordnet.princeton.edu/}

[78] Zelle J. M., Mooney R. J., “Learning to parse database queries using inductive logic programming”, Proc. of the 13th Nat. Conf. on Artificial Intelligence (AAAI–96), 1996, 1050–1055

[79] Luke S., “Zettlemoyer and Michael Collins. Learning to map sentences to logical form: Structured classification with Probabilistic Categorial Grammars”, Proc. of 21th Conf. on Uncertainty in Artificial Intelligence (UAI–2005), Edinburgh, Scotland, 2005, 658–666

[80] Afifi A., Eizen S., Statisticheskii analiz. Podkhod s ispolzovaniem EVM, Per. s angl., Mir, M., 1982, 488 pp. | Zbl

[81] Apresyan Yu. D., Boguslavskii I. M., Iomdin L. L., Lazurskii A. V., Mityushin L. G., Sannikov V. Z., Tsinman L. L., Lingvisticheskii protsessor dlya slozhnykh informatsionnykh sistem, Nauka, M., 1992, 256 pp.

[82] Galushkin A. I., Teoriya neironnykh setei, Kn. 1, IPRZhR, M., 2000, 416 pp.

[83] Zagoruiko N. G., Prikladnye metody analiza dannykh i znanii, Izd-vo IM, Novosibirsk, 1999, 270 pp. | Zbl

[84] Iomdin L. L., Sizov V. G., Tsinman L. L., “Ispolzovanie empiricheskikh vesov pri sintaksicheskom analize”, Obrabotka teksta i kognitivnye tekhnologii, Otechestvo, Kazan, 2001, 64–72

[85] Kobzareva T. Yu., Lakhuti D. G., Nozhov I. M., “Model segmentatsii russkogo predlozheniya”, Trudy konferentsii Dialog' 2001, T. 2, 2001, 185–194

[86] Ledli R., Lasted L., “Meditsinskaya diagnostika i sovremennye metody vybora resheniya”, Meditsinskie problemy v biologii, ed. R. Bellman, Mir, M., 1966, 141–198

[87] Naidenova K., “Reduktsiya zadach mashinnogo obucheniya k approksimatsii zadannoi klassifikatsii na mnozhestve primerov”, Trudy 5-i nats. konf. po iskusstvennomu intellektu, T. 1, Kazan, 1996, 275–279

[88] Finn V. K., “Ob Intellektualnom analize dannykh”, Novosti iskusstvennogo intellekta, 2004, no. 3, 3–18 | Zbl