Foundations of Data Science
Weekly outline

Final Exam
 Friday, January 19, 2024: 15:1518:15. Rooms GCA330 (surnames from A to L) and GCA331 (surnames from M to Z).
 Exam coverage and rules:
 The exam will be open book: all printed material is allowed, but electronic devices are not allowed.
 The exam will cover all class materials including lecture notes, homework, and the contents of live lectures.
 The exam will be open book: all printed material is allowed, but electronic devices are not allowed.
 Special exam office hours:
 Tuesday, Jan 9, 14:0015:00, INR 136 (Marco)
 Wednesday, Jan 10, 14:0015:00, INR 116 (Rüdiger)
 Thursday, Jan 11, 14:0015:00, INR 030 (Thomas)
 Friday, Jan 12, 14:0015:00, INR 130 (Michael)
 Monday, Jan 15, 14:0015:00, INR 030 (Thomas)
 Tuesday, Jan 16, 14:0015:00, INR 136 (Marco)
 Wednesday, Jan 17, 13:0014:00, INR 116 (Rüdiger)
 Thursday, Jan 18, 14:0015:00, INR 130 (Michael)
 Tuesday, Jan 9, 14:0015:00, INR 136 (Marco)
SWITCHtube Channel
We will not make new recording this year. However you can access the videos from last year. The content is largely the same.
Summary
We discuss a set of topics that are important for the understanding of modern data science but that are typically not taught in an introductory ML course. In particular we discuss fundamental ideas and techniques that come from probability, information theory as well as signal processing.
Content
This class presents basic concepts of Information Theory and Signal Processing and their relevance to emerging problems in Data Science and Machine Learning.
A tentative list of topics covered is:
 Information Measures
 Signal Representations
 Detection and Estimation
 Multiarm Bandits
 Distribution Estimation, Property Testing, and Property Estimation
 Exponential Families
 Compression and Dimensionality Reduction
 Information Measures and Generalization Error
Materials
 Lecture Notes (Version of Monday, September 18, 2023). Note: Check for updates on a semiregular basis.
Additional Material: T. M. Cover and J. A. Thomas, Elements of Information Theory (Click to get access to the full PDF via the EPFL library). New York: Wiley. Second Edition, 2006.
 T. Lattimore and C. Szepesvari, Bandit Algorithms
Schedule
Classes:
 Tuesday 11:1513:00 (CE1105)
 Thursday 17:1519:00 (INF1)
Exercise: Tuesday 13:1515:00 (CE1105)
Grading
 If you do not hand in your final exam your overall grade will be NA.
 Otherwise, if we can hold the Midterm Exam (on Thursday, November 16, 2023, 17:1519:00), your grade will be determined based on the following weighted average: 10% for the Homework, 30% for the Midterm Exam, 60% for the Final Exam.
 If we cannot hold the Midterm Exam, your grade will be determined based on the following weighted average: 10% for the Homework, 90% for the Final Exam.
The Final Exam will take place on at some point between January 15, 2024 and February 3, 2024.

Sept 19 : General Introduction ; Probability Review
Sept 21 : Information Measures 
Sept 26 : Information Measures
Sept 28 : Information Measures ; Signal Representations 
Oct 3 : Signal Representations
Oct 5 : Signal Representations 
Oct 10 : Signal Representations
Oct 12 : Signal Representations; Estimation 
Oct 17 : Multiarm Bandits explorethenexploit
Oct 19 : Multiarm Bandits : UCB 
Oct 24 : Multiarm Bandits
Oct 26 : Multiarm Bandits ; Distribution Estimation 
Oct 31 : Distribution Estimation
Nov 2 : Distribution Estimation 
Nov 7 : Distribution Estimation
Nov 9 : Property Testing and Estimation 
Nov 14 : Detection and Estimation
Nov 16 : Midterm Exam 
Nov 21 : Detection and Estimation
Nov 23 : Exponential Families 
Nov 28 : Exponential Families
Nov 30 : Exponential Families 
Dec 5 : Compression
Dec 7 : Compression 
Dec 12 : Compression
Dec 14 : Exploration Bias and Generalization Bounds 
Dec 19 : Generalization Bounds
Dec 21 : Review Session