Paraphrase and Textual Entailment Generation in Czech

Zuzana Nevěřilová

Natural Language Processing Centre, Faculty of Informatics, Masaryk University, Brno, Czech Republic. xpopelk@fi.muni.cz.

Article received on 07/01/2014. ]]> Accepted on 01/02/2014.

Abstract

Paraphrase and textual entailment generation can support natural language processing (NLP) tasks that simulate text understanding, e.g., text summarization, plagiarism detection, or question answering. A paraphrase, i.e., a sentence with the same meaning, conveys a certain piece of information with new words and new syntactic structures. Textual entailment, i.e., an inference that humans will judge most likely true, can employ real-world knowledge in order to make some implicit information explicit. Paraphrases can also be seen as mutual entailments. We present a new system that generates paraphrases and textual entailments from a given text in the Czech language. First, the process is rule-based, i.e., the system analyzes the input text, produces its inner representation, transforms it according to transformation rules, and generates new sentences. Second, the generated sentences are ranked according to a statistical model and only the best ones are output. The decision whether a paraphrase or textual entailment is correct or not is left to humans. For this purpose we designed an annotation game based on a conversation between a detective (the human player) and his assistant (the system). The result of such annotation is a collection of annotated pairs text-hypothesis. Currently, the system and the game are intended to collect data in the Czech language. However, the idea can be applied for other languages. So far, we have collected 3,321 H-T pairs. From these pairs, 1,563 were judged correct (47.06 %), 1,238 (37.28 %) were judged incorrect entailments, and 520 (15.66 %) were judged non-sense or unknown.

Keywords: Games with a purpose, paraphrase, textual entailment, natural language generation.

DESCARGAR ARTÍCULO EN FORMATO PDF

Acknowledgments

This work has been partly supported by the Ministry of Education of CR within the LINDAT-Clarin project LM2010013 and by the Ministry of the Interior of CR within the project VF20102014003.

References

1. Akhmatova, E. (2005). Textual entailment resolution via atomic propositions. In PASCAL: Proceedings of the First Challenges Workshop on Recognising Textual Entailment. Southampton, UK, 61-64.         [ Links ]

2. Androutsopoulos, I. & Malakasiotis, P. (2009). A survey of paraphrasing and textual entailment methods. CoRR, abs/0912.3747.         [ Links ]

3. Bhagat, R. & Hovy, E. (2013). What is a paraphrase? Computational Linguistics, 39(3), 463—472. ISSN 0891-2017. doi:10.1162/COLI_a_00166.         [ Links ]

4. Boyd-Graber, J., Fellbaum, C., Osherson, D., & Schapire, R. (2006). Adding dense, weighted connections to WordNet. In Proceedings of the Third International WordNet Conference GWC-06. Masaryk University in Brno, South Jeju Island, Korea, 29-36.         [ Links ]

5. Chamberlain, J., Kruschwitz, U., & Poesio, M. (2009). Constructing an anaphorically annotated corpus with non-experts: Assessing the quality of collaborative annotations. In Proceedings of the 2009 Workshop on The People's Web Meets NLP: Collaboratively Constructed Semantic Resources, People s Web 09. Association for Computational Linguistics, Stroudsburg, PA, USA. ISBN 978-1932432-55-8, 57-62.         [ Links ]

6. Chklovski, T. (2005). Collecting paraphrase corpora from volunteer contributors. In Proceedings of the 3rd International Conference on Knowledge Capture, K-CAP '05. ACM, New York, NY, USA. ISBN 1-59593-163-5, 115-120. doi:10.1145/1088622.1088644.         [ Links ]

7. Clark, P., Fellbaum, C., & Hobbs, J. R. (2006). The Boeing-Princeton-ISI (BPI) textual entailment test suite. Accessed online 201404-14 from http://www.cs.utexas.edu/~pclark/bpi-test-suite/bpi-rte-knowledge-types.txt.         [ Links ]

8. Dagan, I., Glickman, O., & Magnini, B. (2006). The PASCAL recognising textual entailment challenge. In Quiñonero-Candela, J., Dagan, I., Magnini, B., & d'Alché-Buc, F., editors, Machine Learning Challenges, volume 3944 of Lecture Notes in Computer Science. Springer, p. 177-190.         [ Links ]

9. Fellbaum, C. (1998). WordNet: An Electronic Lexical Database (Language, Speech, and Communication). The MIT Press. ISBN 026206197X.         [ Links ]

10. Grác, M. (2013). Rapid Development of Language Resources. Ph.D. thesis, Masaryk University, Brno, Czech Republic.         [ Links ]

11. Graesser, A. (1981). Prose Comprehension Beyond the Word. Springer-Verlag, New York. ISBN 9783540905448.         [ Links ]

12. Hajič, J., Hajičova, E., Pajas, P., Panevová, J., Sgall, P., & Vidova-Hladká, B. (2001). Prague Dependency Treebank 1.0 (Final Production Label). Linguistic Data Consortium. ISBN 1-58563-212-0. Published: CDROM CAT: LDC2001T10.         [ Links ]

13. Havasi, C., Speer, R., & Alonso, J. (2009). Con-ceptNet: a lexical resource for common sense knowledge. In Nicolov, N., Angelova, G., & Mitkov, R., editors, Recent Advances in Natural Language Processing V, volume 309 of Current Issues in Linguistic Theory. John Benjamins, Amsterdam & Philadelphia, 269-280.         [ Links ]

14. Hlaváčkova, D. & Horák, A. (2005). VerbaLex -new comprehensive lexicon of verb valencies for Czech. In Computer Treatment of Slavic and East European Languages. Slovenský národný korpus. ISBN 80-224-0895-6, 107-115.         [ Links ]

15. Kovář, V., Horák, A., & Jakubíček, M. (2011). Syntactic analysis using finite patterns: A new parsing system for Czech. In Human Language Technology. Challenges for Computer Science and Linguistics, volume November 6-8, 2009. Poznan, Poland, 161-171.         [ Links ]

16. Kučová, L. & Žabokrtský, Z. (2005). Anaphora in Czech: Large data and experiments with automatic anaphora resolution. In Matoušrek, V., Mautner, P., & Pavelka, T., editors, Proceedings of 8th International Conference on Text, Speech and Dialogue, TSD 2005, volume 3658 of Lecture Notes in Computer Science. Springer Berlin Heidelberg. ISBN 978-3-540-28789-6, 93-98. doi:10.1007/11551874_12.         [ Links ]

17. Nevěřilova, Z. (2014). Annotation game for textual entailment evaluation. In Gelbukh, A., editor, 15th International Conference, CICLing 2014, Kath-mandu, Nepal, April 6-12,2014, Proceedings, Part I, volume 8403 of Lecture Notes in Computer Science. Springer. ISBN 978-3-642-54905-2, 340-350.         [ Links ]

18. Pala, K. & Smrž, P. (2004). Building Czech Word-net. Romanian Journal of Information Science and Technology, 2004(7), 79-88.         [ Links ]

19. Palmer, M. S., Dahl, D. A., Schiffman, R. J., Hirschman, L., Linebarger, M., & Dowding, J. (1986). Recovering implicit information. In Proceedings of the 24th Annual Meeting on Association for Computational Linguistics, ACL '86. Association for Computational Linguistics, Stroudsburg, PA, USA, 10-19. doi:10.3115/981131.981135.         [ Links ]

20. Sammons, M., Vydiswaran, V. G. V., & Roth, D. (2010). Ask not what textual entailment can do for you... In Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, Uppsala, Sweden, 1199-1208.         [ Links ]

21. Šmerk, P. (2010). Towards Computational Morphological Analysis of Czech. Ph.D. thesis, Masaryk University in Brno, Brno, Czech Republic.         [ Links ]

22. Šmerk, P. & Hlaváčková, D. (2012). Derivacni an-alyzator ccestiny Derivance [Derivational analyzer for Czech]. [software] accessed online 2014-04-09 from http://nlp.fi.muni.cz/projekty/derivance.         [ Links ]

23. Snow, R., O'Connor, B., Jurafsky, D., & Ng, A. Y. (2008). Cheap and fast—but is it good?: Evaluating non-expert annotations for natural language tasks. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, EMNLP '08. Association for Computational Linguistics, Stroudsburg, PA, USA, 254-263.         [ Links ]

24. Venhuizen, N., Basile, V., Evang, K., & Bos, J. (2013). Gamification for word sense labeling. In Proceedings of the 10th International Conference on Computational Semantics (IWCS 2013) - Short Papers. Potsdam, Germany, 397-403.         [ Links ]

25. Vickrey, D., Bronzan, A., Choi, W., Kumar, A., Turner-Maier, J., Wang, A., & Koller, D. (2008). Online word games for semantic data collection. In EMNLP 08: Proceedings of the Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Morristown, NJ, USA, 533-542.         [ Links ]

26. von Ahn, L. & Dabbish, L. (2008). Designing games with a purpose. Commun. ACM, 51(8), 58-67. ISSN 0001-0782. doi:10.1145/1378704.1378719.         [ Links ]

27. von Ahn, L., Kedia, M., & Blum, M. (2006). Verbosity: a game for collecting common-sense facts. In CHI '06: Proceedings of the SIGCHI conference on Human Factors in computing systems. ACM, New York, NY, USA. ISBN 1-59593-372-7, 75-78. doi: http://doi.acm.org/10.1145/1124772.1124784.         [ Links ]

28. Wang, A., Hoang, C., & Kan, M.-Y. (2013). Perspectives on crowdsourcing annotations for natural language processing. Language Resources and Evaluation, 47(1), 9-31. ISSN 1574-020X. doi: 10.1007/s10579-012-9176-1.         [ Links ]