20605 - MACHINE LEARNING II
Department of Computing Sciences
Course taught in English
Go to class group/s: 31
Course Director:
IGOR PRUENSTER
IGOR PRUENSTER
Suggested background knowledge
Solid knowledge of calculus, linear algebra and probability theory. Good understanding of basic statistical and Machine learning tools (e.g. simple and multiple regression, likelihood-based inferences, optimization). Intermediate level programming (e.g. Python, R or Matlab).
Mission & Content Summary
MISSION
The course introduces students to some frontier research topics in Statistics and Machine Learning, both from a theoretical and applied standpoint. Students are exposed to state of the art methodologies for inferences and prediction and are trained to develop a principled and thoughtful approach towards Machine Learning.
The first part of the course deals with Bayesian Nonparametric theory. We start with foundational and theoretical issues, overview popular models and investigate their inferential implications and showcase computational methodologies and popular applications areas. The second part of the course focuses on modern applied machine learning. We present recent developments and breakthroughs in artificial neural networks and deep learning, reinforcement learning and boosting. An introduction to PyTorch is also provided.
CONTENT SUMMARY
- Foundations of Bayesian Nonparametrics: exchangeability and de Finetti’s representation theorem.
- Nonparametric priors: definition, distributional properties and Bayesian nonparametric models.
- Computational methodologies and sampling algorithms for Bayesian Nonparametrics.
- Monte Carlo methods, gradient-based MCMC, probabilistic programming and software for Bayesian computation.
- Applications: species sampling, mixture models, clustering, topic modeling in document analysis and survival analysis.
- The artificial neural network zoo: feedforward networks, convolutional, recurrent neural networks and sequence-to-sequence models, autoencoders, attention networks and universal transformers, generative adversarial networks.
- Training and regularization: gradient descent, stochastic gradient descent, RMSProp, ADAM, L1 and L2, dropout and early stopping.
- Introduction to PyTorch and GPU usage in ML.
- Reinforcement learning and Boosting.
Intended Learning Outcomes (ILO)
KNOWLEDGE AND UNDERSTANDING
At the end of the course student will be able to...
- Have an overview of cutting-edge statistical and machine learning methods from a theoretical, methodological and applied perspective.
- Understand the assumption and modeling implications underlying machine learning methodologies.
- Decide which method best fits a given problem.
- Understand the foundations of these methods in a way to allow to explain their implementations step by step.
- Understand the data requirements and computational requirements of these methods.
- Understand the problems and pitfalls of testing and applying these methods.
APPLYING KNOWLEDGE AND UNDERSTANDING
At the end of the course student will be able to...
- Design modern models for a given applied problem using Bayesian nonparametrics, Monte Carlo methods, neural networks, reinforcement learning or boosting.
- Understand the results in terms of the characteristics of the chosen method.
Teaching methods
- Face-to-face lectures
- Online lectures
DETAILS
Lectures face-to-face and online.
Assessment methods
| Continuous assessment | Partial exams | General exam | |
|---|---|---|---|
|
x |
ATTENDING AND NOT ATTENDING STUDENTS
The assessment consists in an individual presentation of a research paper selected among a list provided by the instructors. The papers are related to the topics of the lectures and their understanding requires knowledge of the models and methods covered during the course. The presentations are expected to showcase the acquired theoretical, methodological and applied skills and to critically discuss the assumptions, modeling choices and methodologies implemented in the selected paper.
Teaching materials
ATTENDING AND NOT ATTENDING STUDENTS
A reading list of papers suggested by the instructors is provided at the beginning of the course.
Last change 09/05/2022 19:06