Deep Learning neural network models have been successfully applied to natural language processing, and are now changing radically how we interact with machines (Siri, Amazon Alexa, Google Home, Skype translator, Google Translate, or the Google search engine). These models are able to infer a continuous representation for words and sentences, instead of using hand-engineered features as in other machine learning approaches. The seminar will introduce the main deep learning models used in natural language processing, allowing the attendees to gain hands-on understanding and implementation of them in Tensorflow.

This course is a 30 hour in-depth introduction to the main deep learning models used in text processing. It combines theoretical and practical hands-on classes. Attendants will be able to understand and implement the models in Tensorflow.

Student profile

Addressed to professionals, researchers and students who want to understand and apply deep learning techniques to text. The practical part requires basic programming experience, a university-level course in computer science and experience in Python. Basic math skills (algebra or pre-calculus) are also needed.

Contents

Introduction to machine learning and NLP with Tensorflow

Machine learning, Deep learning
Natural Language Processing
A sample NLP task with ML
. Sentiment analysis
. Features
. Logistic Regression
LABORATORY: Sentiment analysis with logistic regression

Multilayer Perceptron

Multiple layers ~ Deep: MLP
Backpropagation and gradients
Learning rate
More regularization
Hyperparameters
LABORATORY: Multilayer Perceptron

Embeddings and Recurrent Neural Networks

Representation learning
Word embeddings
From words to sequences: Recurrent Neural Networks (RNN)
LABORATORY: Recurrent Neural Networks

Seq2seq, Neural Machine Translation and better RNNs

Application of RNN:
. Language Models (sentence encoders)
. Language Generation (sentence decoders)
. Sequence to sequence models and Neural Machine Translation (I)
Problems with gradients in RNN
LSTM and GRU
LABORATORY: Gated Recurrent Units

Attention, Neural machine Translation and Natural Language Inference

Re-thinking seq2seq:
. Attention and memory
. State of the art NMT
Natural Language Inference with siamese networks
LABORATORY: Attention Model

Convolutional neural networks

Convolutional Neural Networks
Deep learning frameworks
Last words
LABORATORY: Convolutional Neural Networks

Instructors

Person 1

Eneko Agirre

Full professor, member of IXA

Person 2

Oier Lopez de la Calle

Postdoc researcher at IXA

Person 3

Olatz Perez de Vinaspre

Postdoc researcher at IXA

Practical details

General information

Part of the Language Analysis and Processing master program
12 theoretical and practical sessions, 30 hours
Scheduled between 22/1/2019 and 7/2/2019. Detailed hourly schedule here
Where: Lab 0.1, Computer science faculty, San Sebastian
Accommodation information (in Basque and Spanish)
Teaching language: English
Capacity: 20 attendants (selected according to CV)
4.5 ECTS credits
Cost: 180 euros

Registration

Please register via email (amaia.lorenzo@ehu.eus) attaching a brief CV.
Use the same address for any enquiry you might have.
Pre-registration open: now to 14th of December

Prerequisite
Basic programming experience, a university-level course in computer science and experience in Python. Basic math skills (algebra or pre-calculus) are also needed.