Theory and Methods for Reinforcement Learning

Weekly outline

• General

  Collapse all Expand all
  • 

    Announcements Forum
  • 

    Logistics File

    Teaching Assistants:
    

    • Luca Viano (Head TA)
    • Leello Dadi
    • Pedro Abranches
    • Yongtao Wu
    • Zhenyu Zhu
    • Andrej Janchevski

    Should you have any question, please send a mail to the head TA (Luca Viano, email = luca.viano@epfl.ch) or post a question on the Moodle discussion board.
    
    

• 20 February - 26 February
  • 

    Lecture 1 File

    An overview of the course

• 27 February - 5 March
  • 

    Lecture 2: Dynamic Programming. File

    MDPs; value and Q functions; value iteration, policy iteration; operator perspectives. Model-free policy-based and value-based methods; Monte Carlo (MC) method and temporal difference (TD) learning.

• 6 March - 12 March
  • 

    Lecture 3: Linear Programming. File

    Primal and Dual LP, ALP, ALP with constraint sampling, primal dual methods, REPS, offline LP methods.
    

• 13 March - 19 March
  • 

    Lecture 4 File

    Policy Gradients Methods.

• 20 March - 26 March
  • 

    Lecture 5 File

    Policy gradient methods II: NPG, Sample Based NPG, TRPO, exploration in policy gradients

  • 

    Dynamic Programming Notebook File

    Exercises on Value Iteration, Policy Iteration, Modified Policy Iteration and Q Learning

• 27 March - 2 April
  • 

    Lecture 6 File

    Imitation Learning: Behavioural cloning, Dagger, MCE-IRL, GAIL, P2IL, IQ-Learn
  • 

    Project Proposal Assignment

    A brief description of the project (2 pages including references) which includes the following:

    1. the names of the project team members

    2. motivation of the projects

    3. formal description of the problem and the goal

    4. references

    5. software and computational resources you will use

    
    

• 3 April - 9 April
  • 

    Lecture 7 File

    Markov Games

    
    

• 17 April - 23 April
  • 

    Lecture 8 File

    Actor Critic based Deep RL: TRPO, Soft Actor Critic.

    Value based Deep RL: DQN, Double DQN, Rainbow.

    Robust RL and IRL.

• 22 May - 28 May
  • 

    Final Project report Assignment

    Dear all,
    
    please upload for your final report by Thursday , Jun 1st at 11:59 PM.

    Please double-check the submission instructions that we uploaded on Moodle during the first week https://moodle.epfl.ch/pluginfile.php/3047530/mod_resource/content/5/syllabus-2023.pdf (page 3) 

    In particular, we expect between 6 and 8 pages in the NeurIPS template https://neurips.cc/Conferences/2022/PaperInformation/StyleFiles

    The suggested structure is 

    1. Abstract
    2. Introduction
    3. Related Work
    4. Approach
    5. Results
    6. Conclusion
    7. References
       

    If you ran experiments, please attach your code as supplementary material, uploading a single zip file containing the main report in pdf format and a folder named supplementary for the attached files.
    It is also possible to upload an Appendix in a separate pdf including it in the same zip file.
    
    The final class is on June 1st when you will be giving a 15 minutes presentation of your project. There is no need to submit the slides you will use at this stage.