#039 - Lena Voita - NLP

2021/1/23

Machine Learning Street Talk (MLST)

Frequently requested episodes will be transcribed first

Shownotes Transcript

ena Voita is a Ph.D. student at the University of Edinburgh and University of Amsterdam. Previously, She was a research scientist at Yandex Research and worked closely with the Yandex Translate team. She still teaches NLP at the Yandex School of Data Analysis. She has created an exciting new NLP course on her website lena-voita.github.io which you folks need to check out! She has one of the most well presented blogs we have ever seen, where she discusses her research in an easily digestable manner. Lena has been investigating many fascinating topics in machine learning and NLP. Today we are going to talk about three of her papers and corresponding blog articles;

Source and Target Contributions to NMT Predictions -- Where she talks about the influential dichotomy between the source and the prefix of neural translation models.

https://arxiv.org/pdf/2010.10907.pdf

https://lena-voita.github.io/posts/source_target_contributions_to_nmt.html

Information-Theoretic Probing with MDL -- Where Lena proposes a technique of evaluating a model using the minimum description length or Kolmogorov complexity of labels given representations rather than something basic like accuracy

https://arxiv.org/pdf/2003.12298.pdf

https://lena-voita.github.io/posts/mdl_probes.html

Evolution of Representations in the Transformer - Lena investigates the evolution of representations of individual tokens in Transformers -- trained with different training objectives (MT, LM, MLM)

https://arxiv.org/abs/1909.01380

https://lena-voita.github.io/posts/emnlp19_evolution.html

Panel Dr. Tim Scarfe, Yannic Kilcher, Sayak Paul

00:00:00 Kenneth Stanley / Greatness can not be planned house keeping

00:21:09 Kilcher intro

00:28:54 Hello Lena

00:29:21 Tim - Lenas NMT paper

00:35:26 Tim - Minimum Description Length / Probe paper

00:40:12 Tim - Evolution of representations

00:46:40 Lenas NLP course

00:49:18 The peppermint tea situation

00:49:28 Main Show Kick Off