20876 - DEEP LEARNING AND REINFORCEMENT LEARNING
Department of Computing Sciences
Course taught in English
Course Director:
ANDREA CELLI
ANDREA CELLI
Conoscenze pregresse consigliate
For an effective learning experience, it is recommended previous programming experience (ideally in Python), good knowledge of calculus and linear algebra, probability and statistics.
Mission e Programma sintetico
MISSION
In this course, we will investigate the algorithmic principles and foundations of two particularly exciting subfields of Machine Learning: deep learning and reinforcement learning. The first part of the course focuses on deep learning, where we develop the conceptual tools to understand what happens when a neural net is trained, and to understand the latest architectural advancements that have proven remarkably effective in practical applications. The second part of the course focuses on sequential decision making, in which an agent interacts with an unknown environment to accomplish a goal. We will start from the multi-armed bandits setting and gradually increase the complexity of the scenarios up to general reinforcement learning problems.
PROGRAMMA SINTETICO
- Part one: optimization for neural net training, overparametrization, attention and memory, GANs, representation learning
- Part two: multi-armed bandits problems, value-based methods, policy-gradient methods, RL with function approximators, Multi-agent RL
Risultati di Apprendimento Attesi (RAA)
CONOSCENZA E COMPRENSIONE
Al termine dell'insegnamento, lo studente sarà in grado di...
- Define the key features of deep learning and reinforcement learning.
- Identify strengths and limitations of deep learning and reinforcement learning algorithms.
- Recognize the connections between optimization and Deep learning/RL.
- Evaluate the main trade-offs in the choice of a technique for a particular problem.
CAPACITA' DI APPLICARE CONOSCENZA E COMPRENSIONE
Al termine dell'insegnamento, lo studente sarà in grado di...
- Formulate and solve machine learning problems using tools from deep-learning and reinforcement-learning
- Independently extend algorithms and theories discussed in class to new problems.
- Understand and use new tools beyond what discussed in class by reading research papers.
Modalità didattiche
- Lezioni frontali
- Esercitazioni (esercizi, banche dati, software etc.)
- Lavori/Assignment di gruppo
DETTAGLI
- Face-to-face lectures focus on the theoretical, methodological, and computational aspects of the topics covered by the course.
- In hands-on exercise sessions students will work on their laptops to implement solutions to relevant case studies. Students will use Python and common machine learning libraries.
- Group assignment: each student is required to participate in a group research-based project and provide a final written report.
Metodi di valutazione dell'apprendimento
| Accertamento in itinere | Prove parziali | Prova generale | |
|---|---|---|---|
|
x | ||
|
x |
STUDENTI FREQUENTANTI E NON FREQUENTANTI
The exam consists of a theory part and a research-based project.
- The theory part consists in exercises and questions to be answered on paper, and it is used to assess the "knowledge and understanding" learning objectives. This contributes to 60% of the final grade.
- The project consists in a group work exploring a research-oriented problem related to the topics of the course. The project contributes to 40% of the final grade. The project is used to assess the "applying knowledge and understanding" learning objectives.
- In order to pass the exam, students must achieve a passing grade in both the theory part and the project part.
Materiali didattici
STUDENTI FREQUENTANTI E NON FREQUENTANTI
- Goodfellow, Ian, Yoshua Bengio, and Aaron Courville. Deep learning. MIT press, 2016
- R.S. Sutton, Richard, A. G. Barto, Reinforcement learning: An introduction, 2018.
Modificato il // :