Abstract: Neural sequence-to-sequence architectures are powerful models for
various NLP tasks, machine translation bewing one of them. We are interested in exploring the representations that are learned by such models when trained on large and diverse multilingual data sets. It is still an open question what kind of linguistic properties are covered and how they are encoded in complex architectures such as a multi-layered transformer architecture, the current state-of-the-art in machine translation and many other tasks. Our main questions include the influence of multilinguality on linguistic abstraction, the traces of specific syntactic and semantic patterns in language representations and the differences of embeddings spaces trained with different objectives. In the talk I will discuss a few of our recent studies as part of the ERC project "Found in Translation" and the additional questions that they raise.
An Analysis of Encoder Representations in Transformer-Based Machine Translation Alessandro Raganato, Jörg Tiedemann Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP https://www.aclweb.org/anthology/W18-5431/
Fixed Encoder Self-Attention Patterns in Transformer-Based Machine Translation Raganato, A., Scherrer, Y. & Tiedemann, J., 1 Nov 2020, Findings of the Association for Computational Linguistics: EMNLP https://www.aclweb.org/anthology/2020.findings-emnlp.49/
Controlling the Imprint of Passivization and Negation in Contextualized Representations Celikkanat, H., Virpioja, S., Tiedemann, J. & Apidianaki, M., 20 Nov 2020, Proceedings of the Third
BlackboxNLP https://www.aclweb.org/anthology/2020.blackboxnlp-1.13/
David Mareček, Hande Celikkanat, Miikka Silfverberg, Vinit Ravishankar, Jörg Tiedemann Are Multilingual Neural Machine Translation Models Better at Capturing Linguistic Features? http://ufal.mff.cuni.cz/pbml/115/art-marecek-et-al.pdf
Speakers: Jörg Tiedemann
Jörg Tiedemann is professor of language technology at the Department of Digital Humanities at the University of Helsinki. He received his PhD in computational linguistics for work on bitext alignment and machine translation from Uppsala University before moving to the University of Groningen for 5 years of post-doctoral research on question answering and information extraction. His main research
interests are connected with massively multilingual data sets and data-driven natural language processing and he currently runs an ERC-funded project on representation learning and natural language understanding.
Affiliation: University of Helsinki