Donnerstag, 07. November 2024, 16:45 - 18:15 iCal
Invited Talk: Terra Blevins
Breaking the Curse of Multilinguality in Language Models
Hörsaal 33 (HS) im Hauptgebäude der Universität Wien
Universitätsring 1, 1010 Wien
Vortrag
Abstract: While language models (LMs) grow larger and gain new capabilities, their performance in non-English languages increasingly lags behind. This is due to the curse of multilinguality, where individual language performance suffers in models trained on many languages. In this talk, I examine how current language models do and don't capture different languages by uncovering how the curse of multilinguality develops during multilingual model training. Building on these insights, I then present two lines of work into breaking this curse. I first discuss best practices for training targeted multilingual models specializing in a single language family. Then, I present a new method, Multilingual Expert Language Models (X-ELM), that expands on the idea of targeted multilingual training and facilitates more equitable massively multilingual language modeling. We show that X-ELMs provide many performance and efficiency benefits over exisiting multilingual modeling approaches, indicating their potential to democratize multilingual NLP.
Bio: Terra Blevins is a postdoctoral researcher at the University of Vienna and an incoming assistant professor at Northeastern University. She holds a Ph.D. in Computer Science from the University of Washington, where she was advised by Luke Zettlemoyer and worked as a visiting researcher at Facebook AI Research (FAIR). Her research focuses on multilingual NLP and analyzing the linguistic knowledge of language models, with the overarching aim of using analysis insights to build better-performing and more equitable multilingual systems.
Veranstalter
Lecture series: Machines that understand? Large Language Models and Artificial Intelligence
Kontakt
Lukas Thoma
Universität Wien
Forschungsgruppe Data Mining and Machine Learning
+43-1-4277-79501
lukas.thoma@univie.ac.at
Erstellt am Donnerstag, 24. Oktober 2024, 16:13
Letzte Änderung am Dienstag, 29. Oktober 2024, 08:58