[Original attention] Neural Machine Translation by Jointly Learning to Align and Translate | AISC

Channel:
Canada LLMs Explained - Aggregate Intellect - AI.SCIENCE
Subscribers:
22,600
Published on ● Video Link: https://www.youtube.com/watch?v=wyQBfi6uOHk


Duration: 1:28:14
5,993 views
70

Toronto Deep Learning Series, 18 October 2018

For slides and more information, visit https://tdls.a-i.science/events/2018-10-18/

Paper Review: https://arxiv.org/abs/1409.0473

Speaker: Xiyang Chen (Rangle.io)

Host: Dessa
Date: Oct 18th, 2018

Neural Machine Translation by Jointly Learning to Align and Translate

Neural machine translation is a recently proposed approach to machine translation. Unlike the traditional statistical machine translation, the neural machine translation aims at building a single neural network that can be jointly tuned to maximize the translation performance. The models proposed recently for neural machine translation often belong to a family of encoder-decoders and consists of an encoder that encodes a source sentence into a fixed-length vector from which a decoder generates a translation. In this paper, we conjecture that the use of a fixed-length vector is a bottleneck in improving the performance of this basic encoder-decoder architecture, and propose to extend this by allowing a model to automatically (soft-)search for parts of a source sentence that are relevant to predicting a target word, without having to form these parts as a hard segment explicitly. With this new approach, we achieve a translation performance comparable to the existing state-of-the-art phrase-based system on the task of English-to-French translation. Furthermore, qualitative analysis reveals that the (soft-)alignments found by the model agree well with our intuition.



Other Videos By LLMs Explained - Aggregate Intellect - AI.SCIENCE


2018-11-30Visualizing Data using t-SNE (discussions) | AISC Foundational
2018-11-27[BERT] Pretranied Deep Bidirectional Transformers for Language Understanding (discussions) | TDLS
2018-11-27[BERT] Pretranied Deep Bidirectional Transformers for Language Understanding (algorithm) | TDLS
2018-11-27Neural Image Caption Generation with Visual Attention (algorithm) | AISC
2018-11-27Neural Image Caption Generation with Visual Attention (discussion) | AISC
2018-11-17PGGAN | Progressive Growing of GANs for Improved Quality, Stability, and Variation (part 2) | AISC
2018-11-16PGGAN | Progressive Growing of GANs for Improved Quality, Stability, and Variation (part 1) | AISC
2018-11-16(Original Paper) Latent Dirichlet Allocation (discussions) | AISC Foundational
2018-11-15(Original Paper) Latent Dirichlet Allocation (algorithm) | AISC Foundational
2018-10-31[Transformer] Attention Is All You Need | AISC Foundational
2018-10-25[Original attention] Neural Machine Translation by Jointly Learning to Align and Translate | AISC
2018-10-16[StackGAN++] Realistic Image Synthesis with Stacked Generative Adversarial Networks | AISC
2018-10-11Bayesian Deep Learning on a Quantum Computer | TDLS Author Speaking
2018-10-02Prediction of Cardiac arrest from physiological signals in the pediatric ICU | TDLS Author Speaking
2018-09-24Junction Tree Variational Autoencoder for Molecular Graph Generation | TDLS
2018-09-19Reconstructing quantum states with generative models | TDLS Author Speaking
2018-09-13All-optical machine learning using diffractive deep neural networks | TDLS
2018-09-05Recurrent Models of Visual Attention | TDLS
2018-08-28Eve: A Gradient Based Optimization Method with Locally and Globally Adaptive Learning Rates | TDLS
2018-08-20TDLS: Large-Scale Unsupervised Deep Representation Learning for Brain Structure
2018-08-14Principles of Riemannian Geometry in Neural Networks | TDLS


Tags:
deep learning
artificial intelligence
machine translation
machine learning
SMT
neural attention
neural machine translation