Insegnamento a.a. 2020-2021

20605 - MACHINE LEARNING II

Department of Decision Sciences

Course taught in English
Go to class group/s: 31
DSBA (6 credits - I sem. - OP  |  FIS/02)
Course Director:
IGOR PRUENSTER

Classes: 31 (I sem.)
Instructors:
Class 31: IGOR PRUENSTER


Suggested background knowledge

Solid knowledge of calculus, linear algebra and probability theory. Good understanding of basic statistical and Machine learning tools (e.g. simple and multiple regression, likelihood-based inferences, optimization). Intermediate level programming (e.g. Python, R or Matlab).

Mission & Content Summary

MISSION

The course introduces students to some frontier research topics in Statistics and Machine Learning, both from a theoretical and applied standpoint. Students are exposed to state of the art methodologies for inferences and prediction and are trained to develop a principled and thoughtful approach towards Machine Learning. The first part of the course deals with Bayesian Nonparametric theory. We start with foundational and theoretical issues, overview popular models and investigate their inferential implications and showcase computational methodologies and popular applications areas. The second part of the course focuses on modern applied machine learning. We present recent developments and breakthroughs in artificial neural networks and deep learning, reinforcement learning and boosting. An introduction to PyTorch is also provided.

CONTENT SUMMARY

  • Foundations of Bayesian Nonparametrics: exchangeability and de Finetti’s representation theorem.
  • Nonparametric priors: definition, distributional properties and Bayesian nonparametric models.
  • Computational methodologies and sampling algorithms for Bayesian Nonparametrics.
  • Monte Carlo methods, gradient-based MCMC, probabilistic programming and software for Bayesian computation.
  • Applications: species sampling, mixture models, clustering, topic modeling in document analysis and survival analysis.
  • The artificial neural network zoo: feedforward networks, convolutional, recurrent neural networks and sequence-to-sequence models, autoencoders, attention networks and universal transformers, generative adversarial networks.
  • Training and regularization: gradient descent, stochastic gradient descent, RMSProp, ADAM, L1 and L2, dropout and early stopping.
  • Introduction to PyTorch and GPU usage in ML.
  • Reinforcement learning and Boosting.

Intended Learning Outcomes (ILO)

KNOWLEDGE AND UNDERSTANDING

At the end of the course student will be able to...
  • Have an overview of cutting-edge statistical and machine learning methods from a theoretical, methodological and applied perspective.
  • Understand the assumption and modeling implications underlying machine learning methodologies.
  • Decide which method best fits a given problem.
  • Understand the foundations of these methods in a way to allow to explain their implementations step by step.
  • Understand the data requirements and computational requirements of these methods.
  • Understand the problems and pitfalls of testing and applying these methods.

APPLYING KNOWLEDGE AND UNDERSTANDING

At the end of the course student will be able to...
  • Design modern models for a given applied problem using Bayesian nonparametrics, Monte Carlo methods, neural networks, reinforcement learning or boosting.
  • Understand the results in terms of the characteristics of the chosen method.

Teaching methods

  • Face-to-face lectures
  • Online lectures

DETAILS

Lectures face-to-face and online.


Assessment methods

  Continuous assessment Partial exams General exam
  • Individual assignment (report, exercise, presentation, project work etc.)
    x

ATTENDING AND NOT ATTENDING STUDENTS

The assessment consists in an individual presentation of a research paper selected among a list provided by the instructors. The papers are related to the topics of the lectures and their understanding requires knowledge of the models and methods covered during the course. The presentations are expected to showcase the acquired theoretical, methodological and applied skills and to critically discuss the assumptions, modeling choices and methodologies implemented in the selected paper.


Teaching materials


ATTENDING AND NOT ATTENDING STUDENTS

A reading list of papers suggested by the instructors is provided at the beginning of the course.

Last change 31/08/2020 22:17